Custom Texture Tool PS v50.1

[Bighead]

Thanks for testing, I was hoping that would be the solution. I always thought TexConv uses the CPU by default, but I was only partially correct.  For most compression types it does, but for BC6H/BC7 it uses the GPU unless disabled. Realizing that BC7 was the only format being corrupted put me on the right path, so thank you for that. But still it makes me wonder why it is working for me. I wonder if it has anything to do with installing the VirtualBox Guest Additions. I have it installed and 3D Acceleration enabled, maybe this is why I'm not seeing corruption; Oracle just might have their shit together.

If you want to try to get hardware acceleration working correctly just for the sake of trying everything or out of curiosity, here is the steps.

I enabled both 2D and 3D acceleration here. Unfortunately this is not enough for it to just work.


Windows VM may need to be started in Safe Mode depending on the version. Once booted Guest Additions ISO can be attached.


After checking Direct3D Support a message will pop up asking to use basic D3D support. I always press NO because I like experimental features.


And that's it. Finish up installing the drivers and reboot the VM. I'm not sure if you installed or, or even doing so will fix corruption for you when using GPU. But I figure now that we know the issue it might be worth seeing if this also fixes it.

I've just about finished replacing DDS Utilities with Nvidia Texture Tools, but it is disappointingly slow when it comes to creating BC7. Like, really really really slow. Converting a 2048x2048 texture took over 3 minutes! Using Compressonator or TexConv it takes less than 20 seconds. So I really don't see it as being ideal for anything other than a backup path for BC7. It is very fast with DXT1/5 and ARGB32 and creates nice textures there, so I suppose it still holds the same purpose as DDS Utilities with the added bonus of being able to create BC7.

[Techie Android]

(11-23-2017, 11:27 AM)Bighead Wrote: Thanks for testing, I was hoping that would be the solution. I always thought TexConv uses the CPU by default, but I was only partially correct.  For most compression types it does, but for BC6H/BC7 it uses the GPU unless disabled. Realizing that BC7 was the only format being corrupted put me on the right path, so thank you for that. But still it makes me wonder why it is working for me. I wonder if it has anything to do with installing the VirtualBox Guest Additions. I have it installed and 3D Acceleration enabled, maybe this is why I'm not seeing corruption; Oracle just might have their shit together.

If you want to try to get hardware acceleration working correctly just for the sake of trying everything or out of curiosity, here is the steps.

I enabled both 2D and 3D acceleration here. Unfortunately this is not enough for it to just work.


Windows VM may need to be started in Safe Mode depending on the version. Once booted Guest Additions ISO can be attached.


After checking Direct3D Support a message will pop up asking to use basic D3D support. I always press NO because I like experimental features.


And that's it. Finish up installing the drivers and reboot the VM. I'm not sure if you installed or, or even doing so will fix corruption for you when using GPU. But I figure now that we know the issue it might be worth seeing if this also fixes it.

I've just about finished replacing DDS Utilities with Nvidia Texture Tools, but it is disappointingly slow when it comes to creating BC7. Like, really really really slow. Converting a 2048x2048 texture took over 3 minutes! Using Compressonator or TexConv it takes less than 20 seconds. So I really don't see it as being ideal for anything other than a backup path for BC7. It is very fast with DXT1/5 and ARGB32 and creates nice textures there, so I suppose it still holds the same purpose as DDS Utilities with the added bonus of being able to create BC7.

Well I've know about that and messed with it a lot in the past but stopped using it because most people say that the 2D/3D acceleration is pretty crappy.
I haven't ever seen a benefit and something I read once said it can cause serious issues.
I'm going to look at it real quick now that you've mentioned it though.
I have thought of looking into GPU pass through but haven't had a chance yet.

[Edit]
Well I'll be darned it worked on all the PS scripts. Actually that makes sense, cause wasn't I using this VM before just fine to convert to DDS BC7? I've had so much change with it I can't keep track. Now what I want do is benchmark CPU conversion to GPU conversion. I'm thinking the VM will be better at utilizing the CPU. Is test file 2 that you sent me fine to use as a benchmark for testing CPU conversion time?

[Edit 2]
Almost forgot to add that the Compressonator failed every time on DDS BC7 and the next one it tried was the Nvidia DDS Utilities, then TexConv which worked.

[Bighead]

(11-23-2017, 12:34 PM)Techie Android Wrote: I haven't ever seen a benefit and something I read once said it can cause serious issues.
I'm going to look at it real quick now that you've mentioned it though.
I have thought of looking into GPU pass through but haven't had a chance yet.

Long ago the GPU acceleration in VirtualBox was somewhat unstable, but I've never seen any serious issues with it. I'm pretty sure they are just wrapping D3D functions to OGL like Wine does. The worst issue I seen was a few years ago. I tried Beyond Good and Evil PC version since it has the demon Tages DRM and wouldn't work past Windows XP. Unfortunately, the graphics corruption was bad. I think it might be much better now, I am able to play the Legend of Heroes Zero/AO games using VirtualBox but haven't tried many other games.

(11-23-2017, 12:34 PM)Techie Android Wrote: Well I'll be darned it worked on all the PS scripts. Actually that makes sense, cause wasn't I using this VM before just fine to convert to DDS BC7? I've had so much change with it I can't keep track. Now what I want do is benchmark CPU conversion to GPU conversion. I'm thinking the VM will be better at utilizing the CPU. Is test file 2 that you sent me fine to use as a benchmark for testing CPU conversion time?

Yes the CPU is always forced in t02. That version is identical to v27.5 aside from that one change. In the next version I will include an option to disable GPU so it can be toggled freely. Luckily I already have this for Waifu2x, so I'll just share that with the convert and rescale options.

(11-23-2017, 12:34 PM)Techie Android Wrote: Almost forgot to add that the Compressonator failed every time on DDS BC7 and the next one it tried was the Nvidia DDS Utilities, then TexConv which worked.

Yeah I figured as much, I haven't actually tested it yet. I'm going to see if there is a way to get Compressonator to work in a VM after I'm finished with everything else. The program claims it uses the CPU by default, so I have no idea why it doesn't work as is.

[Techie Android]

Okay I now checked to see if I could get Compressonator working in a actual Windows installation (Windows 10 on a older laptop of mine AMD CPU/Radeon GPU). I am using the same texture and it isn't working. I'm using the default PS v27.5 with debug on. Compresonator fails and it use TexConv to convert it to DDS BC7. I'll try some more tests tomorrow when I get some time.

[Bighead]

On my main OS I converted two textures to BC7: a 2048x2048 and 96x96. Here GPU acceleration is insanely fast compared to using the CPU. I can now see why it was taking days for you to convert textures. The little texture was quite fast and the big one takes up the bulk of the time.
00:00:13.58 - With
00:03:33.28 - Without

GPU acceleration was actually detrimental when using it in my Windows 7 VM by a few seconds. The results are so close it made me wonder if it's being used at all, but the few seconds gap is consistent every time. Something different seems to be happening with it enabled, or random chance is creating a really good placebo.
00:03:35.39 - With
00:03:28.15 - Without

I ran it several times and the results don't change. This now makes sense why Nvidia Tools are taking so much longer. I discovered it's not using CUDA (and I don't know why yet) so it's been using the CPU. The time it takes about lines up with how long it takes TexConv to create BC7 using only CPU.

So now I tried Windows 8.1 with some scary results. It took over 10 minutes. Surely I thought something is wrong with the VM. Maybe I'm not sharing enough resources. After some tweaking it looks much better.
00:03:40.94 - With
00:03:40.76 - Without

A little bit slower than Windows 7, but there is no fluctuation between enabled and disabled over multiple runs. Results are nearly identical between the two every time, which makes me wonder what's really going on. The toggle I put on the GUI is definitely working. Is the VM really using the GPU in the way we expect? Or really at all? If not why are textures corrupted without guest additions? If so why does everything I listed before fix the corruption? And why does it perform identically to using just CPU? Random chance that creating BC7 using the VMs hardware acceleration and my GTX970 is identical in speed to creating them with my CPU at 4.6GHz? Maybe the VM is actually using only the CPU to do the supposed 2D and 3D acceleration? This is a lot of questions I never had to ask and I don't know enough to answer any of them. I'm curious to see what kind of results you get.

[Techie Android]

(11-24-2017, 02:48 AM)Bighead Wrote: On my main OS I converted two textures to BC7: a 2048x2048 and 96x96. Here GPU acceleration is insanely fast compared to using the CPU. I can now see why it was taking days for you to convert textures. The little texture was quite fast and the big one takes up the bulk of the time.
00:00:13.58 - With
00:03:33.28 - Without

GPU acceleration was actually detrimental when using it in my Windows 7 VM by a few seconds. The results are so close it made me wonder if it's being used at all, but the few seconds gap is consistent every time. Something different seems to be happening with it enabled, or random chance is creating a really good placebo.
00:03:35.39 - With
00:03:28.15 - Without

I ran it several times and the results don't change. This now makes sense why Nvidia Tools are taking so much longer. I discovered it's not using CUDA (and I don't know why yet) so it's been using the CPU. The time it takes about lines up with how long it takes TexConv to create BC7 using only CPU.

So now I tried Windows 8.1 with some scary results. It took over 10 minutes. Surely I thought something is wrong with the VM. Maybe I'm not sharing enough resources. After some tweaking it looks much better.
00:03:40.94 - With
00:03:40.76 - Without

A little bit slower than Windows 7, but there is no fluctuation between enabled and disabled over multiple runs. Results are nearly identical between the two every time, which makes me wonder what's really going on. The toggle I put on the GUI is definitely working. Is the VM really using the GPU in the way we expect? Or really at all? If not why are textures corrupted without guest additions? If so why does everything I listed before fix the corruption? And why does it perform identically to using just CPU? Random chance that creating BC7 using the VMs hardware acceleration and my GTX970 is identical in speed to creating them with my CPU at 4.6GHz? Maybe the VM is actually using only the CPU to do the supposed 2D and 3D acceleration? This is a lot of questions I never had to ask and I don't know enough to answer any of them. I'm curious to see what kind of results you get.

Hm very interesting. So does the Compressonator work for you when converting to DDS BC7? I'm going to be testing the VM and my laptop some more. I'll collect the results and try to narrow this down. My guess right now is that there's a lot going on here with hardware incompatibility (on the laptop) unless the Compressonator has a very small window of operating conditions.

I'm going to research Oracle Virtual Box and see exactly how 2D / 3D acceleration is implemented. And also if it's possible to pass through my GPU to the VM for perfect GPU acceleration. 

[Bighead]

I did manage to get the 32-bit version of Compressonator working in a Windows 8.1 VM that is fully updated with all the latest windows updates. First I tried to load the GUI, but it was complaining that pi-ms-win-crt-runtime-l1-1-0.dll was missing. A bit of research suggested that grabbing all the latest windows updates fixes this (and it did). The time taken using the samples from before is around 00:00:12.02, which is even faster than TexConv on my host OS!? Opening the GUI complained about D3D12 and Vulkan libraries missing, but other than that it also works to create textures. The CLI version seems fully functional and no complaints about these libraries when using the script.

As for a fully updated Windows 7, the GUI crashes because of something QT related (I haven't looked into this issue yet), but the CLI works fine. Again I used the 32-bit version since the 64-bit version does not want to seem to work. More amazing results, the time to compress the same two textures to BC7 was only 00:00:10.52. So it seems like Compressonator is the clear winner when it comes to CPU speed by a landslide. At least, I think its using the CPU as vulkan and D3D12 are not present.

[Techie Android]

@Bighead

The Compressonator just got Linux support, I'm testing it out now.

[Techie Android]

Okay so I've checked the Compressonator (the latest version 64bit) in my Windows 10 VM and it worked!

So here's what I've tested. This was tested using the same 256x256 texture from the other day.

{TexConv}


CPU [5.29s]

GPU [10.14s]


{Compressonator}


CPU/GPU Not sure which the PS script used I was using t01. It was probably the CPU though. [2.44s]

[Techie Android]

I thought I should mention that my VM is setup with exactly half of my system.

From testing I've found this to be a very balanced and stable approach with great performance.

VM Specs:

2 Cores
8 GB RAM

This allows the host to run the VM without running out of any resources.

Over allocating to the VM can and will waste performance on both host and VM.