Off by one texture bug

tursilion · 2015-09-18 04:54:33

I wanted to submit a patch for an off-by-one texture bug that has been in there for a long time, but I am not authorized to use the patch tracker. The issue shows up with Cool Herders (which uses textures to create pseudo sprites and has zero tolerance on the texture coordinates), and frankly not much else. I was working around it by having an emulator-specific build, but I'd like to see the emulator run it properly. :)

It only shows up in one texture mode, and only on the Y axis. This patch reproduces what I had in my codebase to work around it. I don't profess to understand exactly what is wrong, but I suspect it's a rounding error and so adding a fixed point 1 is more a workaround than a fix. However, I tested with all the files I have handy (include the mentioned Dragon Quest IV) and see no new errors, plus Cool Herders now looks like hardware, so I think it's good, at least for now.

--- gfx3d.cpp   2015-09-18 12:35:00.759674100 +0800
+++ gfx3d-new.cpp    2015-09-18 12:07:10.749484100 +0800
@@ -1306,7 +1306,7 @@ static void gfx3d_glTexCoord(s32 val)
        {
                //dragon quest 4 overworld will test this
                last_s = (s32)(((s64)(_s<<12) * mtxCurrent[3][0] + (s64)(_t<<12) * mtxCurrent[3][4] + ((s64)mtxCurrent[3][8]<<12) + ((s64)mtxCurrent[3][12]<<12))>>24);
-               last_t = (s32)(((s64)(_s<<12) * mtxCurrent[3][1] + (s64)(_t<<12) * mtxCurrent[3][5] + ((s64)mtxCurrent[3][9]<<12) + ((s64)mtxCurrent[3][13]<<12))>>24);
+               last_t = (s32)(((s64)(_s<<12) * mtxCurrent[3][1] + (s64)(_t<<12) * mtxCurrent[3][5] + ((s64)mtxCurrent[3][9]<<12) + ((s64)mtxCurrent[3][13]<<12))>>24) + 16;
        }
        else if(texCoordinateTransform == 0)
        {

zeromus · 2015-09-18 05:32:16

Every sourceforge user account is authorized to use the patch tracker.

Thanks for your research, but sorry, at this point, we're not going to add a random numerical hack to fix one game. Especially not one that changes something that is actually 100% correct. The bug is deeper in software rasterizer in the details of the texturing.

We may add a system for per-game hacks at some point, and if we ever do, we'll be sure to add this hack.

Dragon quest 4 I think wasn't written there as a tricky test, just as any test case at all to check for regressions.

tursilion · 2015-09-18 08:27:57

I haven't logged into SourceForge for many years.

That said, just because you don't understand the change doesn't make it a "random numerical hack". It's very specific. You are off by 1. This adds "1.0" and corrects the texture offset. It does so without introducing any regressions that I can see, and I mentioned Dragon Quest only because your comment did, not because it was the only thing I looked at.

What I don't know is whether you are off by exactly 1.0, or if there is some rounding going on in the hardware that the code is instead truncating.

Your suggestion that the bug probably occurs later in the sequence is unlikely - if you mess with the texture coordinates any later than this line of code it affects all texture modes and DOES break other titles.

This line is pretty hairy - it's not impossible that an unexpected truncation is occurring. I suppose we could take this line apart and examine it more closely, but the assertion that this line is 100% correct appears to be made without analysis.

That said, it's your project. Thanks for the consideration.

zeromus · 2015-09-18 18:24:31

I understand the change. It's a random numerical hack. Your visual results were off by one texel. Off by one texel is a common visual result from bugs deep in the rasterizing process. Your numerical hack attempted to undo the off by one texel at an earlier point. The randomness comes from your selection of a place to fix an off by one (there could have been a dozen places) to result in erasing your visual artifacts. The randomness also comes from your selection of 1.0 to fix the bug. These types of texturing bugs are more commonly only broken on some textures and not on others. It's more likely a fractional issue. Of course, choosing some random fraction here would be pretty random. Adding 1.0 is pretty random too. Go read gbatek, theres nothing about adding a 1. The analysis was done at the time when the code was written. Just to be sure I attempted to reproduce the logic on gbatek more precisely (although it's still a bit of a guess what precision is used at various points in the math) and couldnt see any difference in dragon quest 4. I can't find any Cool Herders to test it on.

Basically, it's absurd to try to fix a problem that exists in 100s of games at a codepath which is only followed by dozens of games.

#1 2015-09-18 04:54:33

Off by one texture bug

#2 2015-09-18 05:32:16

Re: Off by one texture bug

#3 2015-09-18 08:27:57

Re: Off by one texture bug

#4 2015-09-18 18:24:31

Re: Off by one texture bug

Board footer