Custom Texture Tool PS v50.1

[Bighead]

I'll take a look at Compressonator soon. I would hope that the command line version is fixed by now, the current incompatibility with the script may just be as simple as some flags changes.

It is normal for DDS files to be larger in size than PNG textures. DDS compression is designed to be most optimal in VRAM. The GPU can use compressed DDS textures directly. File sizes match the same size as they are when loaded into VRAM (+128KB for D3D9 textures, 148KB for D3D11 textures). PNG compression is designed to be most optimal for file size and must be decoded when loaded into VRAM. This is slow and it's why PNG textures can cause stuttering.

If you want a better understanding, it helps to understand how sizes can be calculated. Pixels must be decoded into a format that the GPU understands. To calculate PNG size (or uncompressed DDS) in VRAM, you can multiply the (width * height * channels). Three channels for an RGB texture, four channels when alpha is added for RGBA. Take a 1024x1024 RGBA for example. Even if it's only a 648 KB image, this is not the size in VRAM.

1024 * 1024 * 4 = 4194304 Bytes (4 MB) - This is much larger than 648 KB.

Depending on the block compression type, the resulting size of DDS can vary. For most types, you can assume 1 pixel = 1 byte so simply multiply (width * height) except in BC1 and BC4 where 1 pixel = 0.5 byte so its (width * height / 2). So using the same 1024x1024 texture compressed as either BC1 or BC7...

BC1: 1024 * 1024 / 2 = 524288 Bytes (0.5 MB)
BC7: 1024 * 1024 = 1048576 Bytes (1 MB)

Even though the BC7 texture will be 1MB on disk, which is larger than the PNG at 648KB, it is 1/4 the size in VRAM.

[BlueSwordM]

Thanks a lot for the answers.

[BlueSwordM]

So, I can confirm the new Compressonator tool actually works with the script for some reason.

It's actually a bit faster per thread than the old Compressonator version which is nice.

So, how would I configure the Texture Tool command file to make Compressonator use mutliple threads along with SIMD acceleration/openGL-Vulkan GPU acceleration?

Because I see this in the file itself.

function CompressonatorCLI([string]$ImageFile, [string[]]$Arguments, [string]$OutputFile)
{
# A block of code is needed to pass to the job.
$ThreadedScript = {

# Everything that is passed to Compressonator needs to be copied over.
Param ([string]$Compressonator, [string]$ImageFile, [string[]]$Arguments, [string]$OutputFile)

# Run Compressonator in a new thread.
& $Compressonator $Arguments $ImageFile $OutputFile
}
# Start the job to create the image.
$Job = Start-Job -Name 'ThreadedScript' -Scriptblock $ThreadedScript -ArgumentList $Compressonator,$ImageFile,$Arguments,$OutputFile

# Lock certain elements of the script into a loop while the job runs.
StartJobLoop -Name 'ThreadedScript' -Program 'Compressonator'
}

I'm currently looking in the CLI tool itself because it is quite nice.

[Bighead]

That function is only used to pass commands to and call Compressonator.exe. Don't be mislead by the "new thread" comment, the job loop itself waits for the Compressonator "thread" to finish so calling another one would only mess stuff up. The script technically only uses two threads, the main one for updating the GUI, and a second one for whatever program is called. It would require quite a lot of changes to make it accommodate more than one instance. I know this isn't your question but I figured I should mention it.

As for your question, let me start by saying my code is awful. If what you are looking for is to add more command line options, the function you want to search for is DDSTool_Compressonator, which sets up data for the CompressonatorCLI function. Unlike batch, commands are passed to the program slightly different. Just writing a single line out with commands usually doesn't work. The method I use is to create an array of commands. If you look through the function a bit you will see...

Code:

# Build a list of default arguments to feed into Compressonator.
$ArgumentList = [Collections.ArrayList]@('-fd', $DDSType, '-Performance', '1', '-noprogress', '-nomipmap')

This is where command line options are assembled. You can simply add them to the collection above (surrounded by quotes, separated by commas). Or you can add them using something like.

Code:

$ArgumentList.Add('command1')
$ArgumentList.Add('command2')

Arguments are fed in the order they are assembled, so be careful not to interrupt a two part command like (-Performance 1).

Edit: I also thought I should mention to not remove the "-nomipmap" flag even if you want mipmaps. I have my own method of generating mipmaps that provides consistent results across all programs and doesn't add any junk data to the header. Removing that flag may result in Compressonator adding its own mipmaps, then the script attempting to generate them. I have no idea what the results would be; my guess is lots of errors.

[Ivan89el]

Generate png mipmaps not correct: https://i.imgur.com/EXdUOFL.png

[Bighead]

(09-05-2020, 05:10 AM)Ivan89el Wrote: Generate png mipmaps not correct: https://i.imgur.com/EXdUOFL.png

That screenshot isn't very helpful especially without a comparison. There could be a few different things going on. Without seeing the original textures it's hard to know what that is. Some games use arbitrary mipmaps, some games do tricks with alpha, some use multiple textures for a single asset. While auto generating mipmaps works in most cases, this is probably one where it won't.

Edit: I should add, if the dumped mipmaps are different than the base texture (arbitrary), the script will use them if you include them when generating new mipmaps. But you must have "Arbitrary Mipmaps" enabled when dumping, and possibly "GPU Texture Decoding" disabled (someone correct me if I'm wrong and this isn't the case anymore). If arbitrary mipmaps are dumped correctly the mipmaps will have an "_arb" suffix in their filename.

[BlueSwordM]

Anyway, thanks a lot for your help again. regarding your command lines.
However, it seems like it doesn't actually work for some reason.

Say, if I want to put this:

Code:

# Build a list of default arguments to feed into Compressonator.
$ArgumentList = [Collections.ArrayList]@('-fd', $DDSType, '-Performance', '1', '-EncodeWith', 'HPC', '-noprogress', '-nomipmap')

or

Code:

# Build a list of default arguments to feed into Compressonator.
$ArgumentList = [Collections.ArrayList]@('-fd', $DDSType, '-Performance', '1', '-EncodeWith', 'OCL', '-noprogress', '-nomipmap')

Or even without the -Performance 1 line, it doesn't seem to be actually working, unless I missed something.

It doesn't actually seem to work, even by making sure it points to the correct Compressonator build binary, and making sure it works by itself(already did, it works fine).

Is there anything in your code that might be conflicting with the parameters, or did I add them incorrectly?

[Bighead]

It looks like CompressonatorCLI is still not working correctly or something has changed. I finally stopped being lazy, downloaded the latest version, linked it to the script, and gave it a shot... and it's silently failing and using TexConv instead. You can verify this if you enable the console in debug options.

Spoiler: (Show Spoiler)

It should also show which program is being used in the "message" section of the texture list.

Spoiler: (Show Spoiler)

I'm going to have to investigate further, there's a good chance there is still something weird going on. Program specific errors don't show up in the console because they are ran in a separate PS instance so it's hard to tell why it's failing specifically. It's possible to import errors into the main instance with "Get-Job" and "Write-Output" commands, but currently I only do that for ESRGAN and SFTGAN to catch "out of VRAM" errors. It's been some time since I last tried newer versions of Compressonator, but I remember having issues with both windows command line (CMD.exe), batch scripts, and powershell. Malkierian made an issue a few months back, but so far not much has come out of it. I'll make some modifications so I can test the current version a bit to see if this issue is still valid.
https://github.com/GPUOpen-Tools/compressonator/issues/71

[BlueSwordM]

You're right.
I'm getting a similar but worse problem. It doesn't even manage to compress it for some reason.
If I force it manually via the command line, it actually works fine(except for GPU encoding, which only seems to work with the GUI).
I'm going to do more investigation regarding this issue.
Thanks a lot for your interest regarding this issue.

[Ivan89el]

(09-05-2020, 07:24 AM)Bighead Wrote: That screenshot isn't very helpful especially without a comparison. There could be a few different things going on. Without seeing the original textures it's hard to know what that is. Some games use arbitrary mipmaps, some games do tricks with alpha, some use multiple textures for a single asset. While auto generating mipmaps works in most cases, this is probably one where it won't.

Edit: I should add, if the dumped mipmaps are different than the base texture (arbitrary), the script will use them if you include them when generating new mipmaps. But you must have "Arbitrary Mipmaps" enabled when dumping, and possibly "GPU Texture Decoding" disabled (someone correct me if I'm wrong and this isn't the case anymore). If arbitrary mipmaps are dumped correctly the mipmaps will have an "_arb" suffix in their filename.

The problem is converting pngs. My Pngs (nomipmap):  https://i.imgur.com/pbIHkl3.png
My pngs are converted to pngs Custom Texture Tool PS v46. 2 (nomipmap) https://i.imgur.com/afXwvdi.png