Custom Texture Tool PS v50.1

[Techie Android]

@Bighead

Nice update!

I've been wondering if your PS tool can check if png textures have been compressed to the maximum level losslessly, and if not then compress them further.

[Bighead]

Sorry I missed this post somehow. It's possible to do this with the "Optimize PNG Textures With OptiPNG" option, which of course requires OptiPNG. Sometimes Dolphin will not like the result and fail to load the texture when any color channel (including alpha) is less than 8-bit, or if the result becomes a paletted image (indexed). The script will make sure not to keep these results, so technically not every single image will be "max compressed", but the ones that won't cause issues can. Not every indexed image, nor every image with a color channel that has less than 8-bit depth will fail to load in Dolphin, but it's not possible to know which ones will so I block them all.

[Techie Android]

(01-11-2018, 01:34 PM)Bighead Wrote: Sorry I missed this post somehow. It's possible to do this with the "Optimize PNG Textures With OptiPNG" option, which of course requires OptiPNG. Sometimes Dolphin will not like the result and fail to load the texture when any color channel (including alpha) is less than 8-bit, or if the result becomes a paletted image (indexed). The script will make sure not to keep these results, so technically not every single image will be "max compressed", but the ones that won't cause issues can. Not every indexed image, nor every image with a color channel that has less than 8-bit depth will fail to load in Dolphin, but it's not possible to know which ones will so I block them all.

Ah okay. I'm not going to worry about it, I'm already too paranoid about quality and compression. :/

[Bighead]

OptiPNG won't reduce image quality only file size, it will only create results that have identical pixels. OptiPNG just tries to find the best possible way to get that result. If the pixels change at all, then its a bug in OptiPNG.

Edit: Also, it can create some images using a "non-standard" sRGB profile. I don't know too much about the details of that, but when you run PNG images through the script, some programs that use libpng will make this complaint. But, I've never seen any issue actually arise from this.

[Techie Android]

(01-11-2018, 02:28 PM)Bighead Wrote: OptiPNG won't reduce image quality only file size, it will only create results that have identical pixels. OptiPNG just tries to find the best possible way to get that result. If the pixels change at all, then its a bug in OptiPNG.

Edit: Also, it can create some images using a "non-standard" sRGB profile. I don't know too much about the details of that, but when you run PNG images through the script, some programs that use libpng will make this complaint. But, I've never seen any issue actually arise from this.

Yeah, I just want to wait right now. When I have time, I'll probably just grab the source code and compile it with the latest version of libpng, and mess around with it some.

[Techie Android]

https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-5.1

https://github.com/PowerShell/PowerShell

Anything change here that might mess up your PS script?

I'm going to be playing around with it.

[Techie Android]

I've made a interesting discovery.

Apparently compared to the script I just wrote up to convert high quality DDS BC7 yours is producing lesser quality ones, let me explain.

The script I made is simple and does the job and uses "-Quality 1 -Performance 0".

Then I tested it using "-Quality 0 -Performance 1".

Finally tested in the latest version of your PS script on Windows 10 with the exact same compressonator version (by exact I mean the latest which included GNU/Linux support).

Here are the results, the quality degradation is best seen at the borders of the colors:

Spoiler: (Show Spoiler)

Also when I tried to convert the DDS BC7 textures back to png using your PS script they got messed up somehow (all of them looked like this):

Spoiler: (Show Spoiler)

NOTE this was the ONLY instance I used the PS script after updating to PS 6.0 so it might be that it was a result of that, however the other tests involving conversion from png to DDS BC7 were done before installation of PS 6.0.

All of my testing files can be downloaded here.

[Bighead]

(01-12-2018, 12:37 PM)Techie Android Wrote: https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-5.1

https://github.com/PowerShell/PowerShell

Anything change here that might mess up your PS script?

I'm going to be playing around with it.

I doubt a newer version of PowerShell should have any issues as long as it retains all backwards compatibility. I haven't used any features added from beyond v2 so it remains compatible with Windows 7, with the one exception of centering the console on launch which requires v3. So this means Windows 8, or Win7 users updating PowerShell (which I can't expect anyone to do).

(01-12-2018, 02:57 PM)Techie Android Wrote: I've made a interesting discovery.

Apparently compared to the script I just wrote up to convert high quality DDS BC7 yours is producing lesser quality ones, let me explain.

The script I made is simple and does the job and uses "-Quality 1 -Performance 0".

Then I tested it using "-Quality 0 -Performance 1".

Very nice info, thanks for the heads up! That is a huge difference for the edges, I can't believe I overlooked these options. The script is most likely just using the defaults since I did not define them. I'm going to have to try these out, do you know what the upper limit to either of these is? The documentation and /help for Compressonator itself is pretty vague and doesn't really explain much more than "sets quality" and "sets performance". I'm curious if higher than 1 quality is even better, or if increasing performance also theoretically decreases quality, or at least how performance is decided. Either way, this is a nice upgrade that I didn't even know was possible, and greatly reduces (nearly eliminates) the quality penalty of not using an uncompressed format like ARGB32.

(01-12-2018, 02:57 PM)Techie Android Wrote: Also when I tried to convert the DDS BC7 textures back to png using your PS script they got messed up somehow (all of them looked like this):

Spoiler: (Show Spoiler)

Yeah I know this bug. TexConv does this when creating PNG textures from BC7, and it's the only program I know that can do it at all. I thought I worked around this in some weird way that I don't remember. It may have involved just running the "too bright" PNG through ImageMagick, recreating it and forcing RGBA (color type 6), which somehow fixed it. I'll have to dig back into that mess to see if I messed something up along the way.

This is all good timing as my mind is currently somewhat "code oriented" again and I planned on making a small update in near future. Recently I've been rewriting the upscaling filter logic so its just overall better, and can upscale included mipmaps as well for either PNG or DDS images. My recent endeavors with arbitrary mipmaps made me realize it lacks this feature and just creates all mipmaps from the base image.

Edit: I tested and can reproduce the BC7>PNG issue, but, I am indeed using the fix I mentioned above and it seems to work for me. Just out of curiosity which version of ImageMagick are you using? Maybe it doesn't work in newer versions, I'll update and try again.

As for the quality setting, it looks like anything above 2 is basically just 2, which is higher quality in some aspects, but worse in others. Setting "1" based on this example seems ideal, I'll test more textures later.

[Admentus]

Awesome to hear that the quality of DDS textures can even go further. I suppose a new update in imminent? Perfect timing too, I should still finish porting Clock Town for Majora's Mask.

[Techie Android]

(01-12-2018, 06:33 PM)Bighead Wrote: I doubt a newer version of PowerShell should have any issues as long as it retains all backwards compatibility. I haven't used any features added from beyond v2 so it remains compatible with Windows 7, with the one exception of centering the console on launch which requires v3. So this means Windows 8, or Win7 users updating PowerShell (which I can't expect anyone to do).


Very nice info, thanks for the heads up! That is a huge difference for the edges, I can't believe I overlooked these options. The script is most likely just using the defaults since I did not define them. I'm going to have to try these out, do you know what the upper limit to either of these is? The documentation and /help for Compressonator itself is pretty vague and doesn't really explain much more than "sets quality" and "sets performance". I'm curious if higher than 1 quality is even better, or if increasing performance also theoretically decreases quality, or at least how performance is decided. Either way, this is a nice upgrade that I didn't even know was possible, and greatly reduces (nearly eliminates) the quality penalty of not using an uncompressed format like ARGB32.


Yeah I know this bug. TexConv does this when creating PNG textures from BC7, and it's the only program I know that can do it at all. I thought I worked around this in some weird way that I don't remember. It may have involved just running the "too bright" PNG through ImageMagick, recreating it and forcing RGBA (color type 6), which somehow fixed it. I'll have to dig back into that mess to see if I messed something up along the way.

This is all good timing as my mind is currently somewhat "code oriented" again and I planned on making a small update in near future. Recently I've been rewriting the upscaling filter logic so its just overall better, and can upscale included mipmaps as well for either PNG or DDS images. My recent endeavors with arbitrary mipmaps made me realize it lacks this feature and just creates all mipmaps from the base image.

Edit: I tested and can reproduce the BC7>PNG issue, but, I am indeed using the fix I mentioned above and it seems to work for me. Just out of curiosity which version of ImageMagick are you using? Maybe it doesn't work in newer versions, I'll update and try again.

As for the quality setting, it looks like anything above 2 is basically just 2, which is higher quality in some aspects, but worse in others. Setting "1" based on this example seems ideal, I'll test more textures later.

Interesting. I didn't think setting the quality to anything above 1 would change things. Here's what they say about the quality and performance settings: 

Quote:quality - Quality of encoding. This value ranges between 0.0 and 1.0. (Valid only for BC7 in this release) default is 0.01, 0.0 gives the fastest, lowest quality encoding, 1.0 is the slowest, highest quality encoding. In general even quality level 0.0 will give very good results on the vast majority of images.images. quality settings may be needed for some difficult images (e.g. normal maps) to give good results. Encoding time will increase significantly at high quality levels. Quality levels around 0.8 will give very close to the highest possible quality, increasing the level above this will cause large increases in encoding time for very marginal gains in quality.

performance - Performance of encoding. This value ranges between 0.0 and 1.0. (Valid only for BC7 in this release) Typical default is 1.0. Encoding time can be reduced by increasing this value for a given Quality level. Lower values will improve overall quality with optimal setting been performed at a value of 0.

This information can be found in the source code: https://github.com/GPUOpen-Tools/Compressonator/blob/master/Compressonator/Header/Compressonator.h