Tooltip:
"Enables reinterpreting the data inside the EFB when the pixel format changes.
Some games depend on this function for certain effects, so enable it if you're having glitches."
...
The most common (non-multisampled) EFB formats used are RGB8 (24 bits per pixel using 8 bits per component), RGBA6 (24 bits per pixel using 6 bits per component) and RGB565 (16 bits per pixel using 5 bits for red/blue and 6 bits for green). Sometimes, games switch between two formats during rendering, Super Mario Sunshine for example switches forth and back between RGB8 and RGBA6 (the only two switches which are implemented right now btw).
The GC/Wii don't do any conversion of the existing data inside the EFB (nor will it be cleared, the data just stays there), the byte data is just kept as it is. E.g. if you switch from RGB8 to RGBA6, the pixels will be "reinterpreted" from R[abcdefgh]G[ijklmnop]B[qrstuvwx] to R[abcdef]G[ghijkl]B[mnopqr]A[stuvwx]. As an example, if the source RGB8 pixel is 0x010101 (R value 1; G value 1; B value 1; no alpha channel), the destination RGBA6 pixel STILL has the value 0x010101 but this maps to 0b[000000][010000][000100][000001] (R value 0; G value 16; B value 4; A value 1; Not sure if I got the component order right, but you get the idea).
Now in Dolphin, we can't emulate this behavior directly: Firstly, the RGBA6 format isn't even supported by Direct3D (not sure about OGL, but I guess it's not supported there either) and second, you can only change texture formats by destroying the current texture and creating a new one (i.e. losing all of the texture's contents). Thus, we are always using an 32-bit RGBA8 buffer for storing the EFB data, which gives us some trouble. Video_Software doesn't have this problem since it just uses the native format (doing sw rendering anyway, so no limitations either
). Apparently no one really noticed since it's somewhat hackish coding to change render target formats DURING rendering and even depending upon the correct hardware behavior for the render results. However, during my ClearScreen changes I noticed that it introduced a little glitch in Super Mario Sunshine. Upon further investigation I found out that the efb format change behavior was the culprit.
So what are we doing when the EFB format change emulation is enabled? Pretty simple actually. We're now keeping two render buffers (two EFBs so to speak) and when the game requests a format change, we copy the originial render buffer to the second (using a special conversion pixel shader which takes care of the byte data reinterpretation) and set the second texture as the active render target. It's quite slow if the game makes heavy use of format changes, which is why I added an option for it (esp. since most games don't depend on this at all but still do many format changes).
Turns out it fixed quite a few other games than just Sunshine by the way
...
Erm, damnit wrote that much again... have fun reading.