(11-17-2016, 12:58 AM)JosJuice Wrote: [ -> ]Only the option is new, not the feature. Before the option was added, 24-bit color was always used.
Ohh well. My bad. If I understand correctly, the GameCube uses 6-bits of precision for the alpha channel (which should be the EFB channel and the three RGB color channels) while Dolphin actually used 8-bits for each channel. So disabling the option (which is the new option right?) allows Dolphin to actually use 6-bits of precision for each channel as expected. Using 8-bits instead should decrease the color banding for increased quality. If I am correctly so far. So with the option enabled Dolphin uses 24-bit RGB colors (which would be 8-bit per RGB color channel) and the remaining 8-bit for the alpha channel. I believe I am somehow messing up my general understanding of the color use...
(11-17-2016, 01:14 AM)Admentus Wrote: [ -> ]If I understand correctly, the GameCube uses 6-bits of precision for the alpha channel (which should be the EFB channel and the three RGB color channels) while Dolphin actually used 8-bits for each channel.
The alpha channel is separate from the three RGB channels. (I'm not sure what you mean by the EFB channel.) The GameCube uses 6 bits for all four channels, and Dolphin used to use 8 bits for all four channels. When the option was added, Dolphin was changed to always use 6 bits for the alpha channel and either 6 or 8 bits (depending on the option) for the RGB channels.
Ed, Edd n Eddy will run into issues if the alpha channel is 8 bits.
(11-17-2016, 01:14 AM)Admentus Wrote: [ -> ]So disabling the option (which is the new option right?) allows Dolphin to actually use 6-bits of precision for each channel as expected. Using 8-bits instead should decrease the color banding for increased quality. If I am correctly so far. So with the option enabled Dolphin uses 24-bit RGB colors (which would be 8-bit per RGB color channel) and the remaining 8-bit for the alpha channel. I believe I am somehow messing up my general understanding of the color use...
Yes, except Dolphin's alpha channel is always 6-bit if you're using a version where the option exists.
I was checking this article:
https://en.dolphin-emu.org/blog/2016/11/01/dolphin-progress-report-october-2016/#cuthere. It described that the alpha channel is part of the framebuffer (EFB). I must have misphrased I guess. It was quite interesting to read up to. So each RGB color is a channel, but if the EFB is not a channel, which is the fourth channel than?
Quote:So what went wrong with storing the stencil in the alpha channel? Well, it turns out that on the GameCube you don't get the extra alpha channel for free. The alpha channel is part of the framebuffer (EFB) together with the red, green and blue color channels (RGB) and the GameCube only has 24-bit per pixel in the framebuffer. So to use the full 24-bit color range of your monitor it has to use everything for the RGB color channels. When you want to enable the alpha channel it has to make room for that by only assigning 6 bits to each color channel, which leaves another 6 bits free for the alpha channel. Dolphin on the other hand actually has 32-bit per pixel in the framebuffer, so it can use 24-bit colors and still have 8-bit left for the alpha channel.
But using 8-bit precision when the game only expects 6-bit will result in rounding errors at some point and it turns out that assumption finally broke something with this game. Despite being somewhat maligned, Dolphin's software renderer came into incredible use with this issue because it rendered everything correctly. Since the software backend was written to be as accurate as possible it was already using 6-bit for each channel in the framebuffer. Being able to compare the broken implementation to a working one proved that we finally found the problem.
The fourth channel is the alpha channel. The EFB stores all four channels, but the EFB itself is not a channel.
EDIT: The article just reminded me that I've missed a detail. Dolphin doesn't use 6-bit alpha channels for everything now, but it does always use 6-bit alpha channels when storing to the EFB. (Maybe the same is true of the RGB channels when the 24-bit option is off? I'm not sure, since haven't worked on the graphics part of Dolphin.)