Custom Texture Tool PS v50.1

[Matrix2525]

Hey there Bighead.  Firstly, I want to say I'm amazed at how relatively quickly you've managed to add ESRGAN and SFTGAN support to your texture tool, even with all the hiccups left to iron out.  I've been jumping down the ESRGAN rabbit hole myself recently while working on one of my texture packs.  It may not help any of your current issues, but I thought I would share some useful information and resources I've come across. First up, here's a link to the initial reddit thread (for context) that started the ball rolling for me: Metroid Prime 2 upscaled with a neural network

The important part is the OP's link to a very useful set of scripts and tools by rapka: https://github.com/rapka/dolphin-textures

This is the primary toolset I've been using to upscale with.  The big points are that rapka uses his own forked branch of ESRGAN in conjunction with the manga109 model and his own scripts geared to upscale textures for dolphin.  Out of those scripts, the biggest feature I've come across is scripts for handling images with alpha transparency.  They separate, upscale, and re-integrate the alpha channel into the upscaled transparent image.  I've also found that ESRGAN's ability to upscale is solely dependent on the model it uses, and the manga109 model is one of the most impressive I've seen so far for textures.  I put together a quick comparison between models used on a 128x128 texture (manga109, PSNR_x4, ESRGAN_x4, and an interpolation between PSNR and ESRGAN), plus an upscaled text sheet with transparency:

Spoiler: (Show Spoiler)

I hope these links might be of use, or at least interest.  Unforunately, I don't have any helpful advice for the texture size/VRAM issue, other than to confirm that trying to upscale a 1024x1024 texture fails for me with a GTX 1080 and 8 GB of vram. 512x512 is my limit (I haven't tested any sizes in-between).

[Bighead]

Oh wow, thanks for the info/links! I haven't attempted to tackle anything yet but I'm sure it will all be very useful when I do.

[Bighead]

A quick update to v39, resource pack zips are not working so the script once again requires 7-Zip.
http://www.mediafire.com/file/0glwlmbm9gzu1w6/Custom_Texture_Tool_PS_v39.5.zip

ESRGAN/SFTGAN version with resource pack generation working. Does require 7-Zip.
http://www.mediafire.com/file/nx5d1f7rnyifff9/Custom_Texture_Tool_PS_v40.0.b5.zip

Do not use the current beta with ESRGAN/SFTGAN to create resource packs. I'm not sure if v40 will require 7-Zip or not. (Edit: It will.) I would like to try to figure out if there is a solution but there might not be a solution. For some reason .NET zip files are adding a few bytes per file when defining "No Compression". The difference makes it so it doesn't work in Dolphin (issue mentioned by ewzzy here). This doesn't happen when using WinRAR or 7-Zip.

Spoiler: (Show Spoiler)

Found some info on why the 5 bytes are added is mentioned here and here. Looks like there may not be a solution, as it's a limitation of the class. It doesn't create zip's with "Store" mode, but rather compression level 0.

[Bighead]

This test version allows creating much higher resolution texture with ESRGAN/SFTGAN on lesser hardware.
http://www.mediafire.com/file/chads35uv5mcwh3/Custom_Texture_Tool_PS_v40.0.b6.zip

A list of important things to note.

Spoiler: (Show Spoiler)

• This extends the capabilities of ESRGAN and SFTGAN, allowing low-mid range GPUs to create higher resolution images. This is done by carving an image into segments, applying the upscaler, then splicing the upscaled pieces together. There is pixel overlap accounted for in each segment so they remain seamless.

• All my tests have been on a GTX 970 with 4GB VRAM, and I have been able to upscale 1024x1024 images to 4096x4096 with no issue. While it's great that the limitation has been removed, there is of course some pros and cons.

• The obvious pro is that we can create much larger images. Since images are created in segments, there is no limit on how large an image can be created.

• The most notable con is that it that the upscaling process can take a very long time. Exponentially longer the larger the image.

• In the current implementation, the theoretical limit of input image resolution should be 2048x2048 before stuff starts to break. This means the maximum upscale you can get with this method at the moment is 8192x8192. This limit is artificial, it can easily be increased by increasing the number of possible segments.

• I will increase the limit next version, right now I just want to keep dimensions within sane values for testing purposes.

• ESRGAN and SFTGAN are very different despite their similar names. SFTGAN is very, very resource hungry, and takes about 4x longer than ESRGAN for the same resolution.

• When ESRGAN is fed an image, the result is 16x larger (4x dimensions). I am assuming the upscale factor is determined by the models. I have yet to look into additional ones floating around on the web.

• SFTGAN requires the image to be upscaled by a factor of 4x before it is processed. This means that compared to ESRGAN, the script has to create 4x as many segments. When upscaling is done, for now I have chosen the nearest neighbor (point) filter since it was mentioned on the blog post here. I will eventually test other filters too, and if it proves to be beneficial I may add options to choose which filter is used.

• To further explain why STFGAN is slow, during the upscaling process, it also splits each image into 3 additional images. Upscaling an image of 1024x1024 to 4096x4096 requires splitting the image into 256 segments (!) to keep the RAM limit within a 4GB range. On my GTX970, this took nearly 50 minutes.

• It would be wise to test this on a smaller version of your images, find results that you like, then upscale a larger version when you think you have found a "final result".

I'll post a brief description of how it works below:

Spoiler: (Show Spoiler)

   
This is SFTGAN example so image is upscaled 4x with nearest neighbor (point) filter. Image is then fed to the ImageMagick which tiles the texture in a 3x3 grid.



Upscaling this would be wasteful, only a few neighboring pixels are needed. All but 16 rows and columns on each side (total of 32) are cropped away.



Now there are seamless pixels for the upscaler to work with, and a grid I can align to (although this texture is a bad example since it is not seamless).



The image is then segmented. The dimensions of segments are calculated based on dimensions of input image, the maximum size for a segment is 288x288 (256x256 +32 extra seamless pixels). Larger dimensions require more segments and increases the time it takes exponentially.



In this example, there are two red squares. This is the first segment. The outer border holds temporary pixels for the upscaler. After being sent through ESRGAN/SFTGAN, these pixels are then cropped away around the center border. The blue squares would be the next segment in the process.



At the end, all the filtered/cropped segments are combined into the final image.

Hopefully it works well for everyone. I wanted to at least get this out since I still have quite a bit I want to do before finishing this version up. I'm hoping to implement the same concept for keeping alpha channels as rapka did in his scripts. I am also going to switch to his fork of ESRGAN for the auto-install since he has the models already included + additional ones that aren't included on the main repository.

I do see one issue with keeping alpha: time. These filters are already pretty intense, and extracting the alpha channel into a separate image means doubling the time it takes for each image. SFTGAN also creates an additional layer of complexity since it can't even work with images with an alpha channel. In the script's current state, it strips the alpha channel before feeding it into SFTGAN. In short, if I can actually pull it off, I'll probably make it an optional thing.

[Bighead]

This will probably be the last one before releasing. (Edit: I lied again, I found two issues with SFTGAN that needed to be fixed.)
http://www.mediafire.com/file/x9280xvw0863avj/Custom+Texture+Tool+PS+v40.0.b7.zip
http://www.mediafire.com/file/59wixd053mface3/Custom+Texture+Tool+PS+v40.0.b8.zip

ESRGAN/SFTGAN features are complete, and they seem to work. I'll have to test it more of course.



Alpha channels can now be kept, but it will take twice as long to upscale since 2 images are processed. I also made it possible to enable/disable processing images in segments for those with more powerful hardware that don't need the option. And since adding pixels can increase the time it takes to upscale, its possible to configure the number of extra pixels. From my very limited testing, 24 pixels seems to be enough to eliminate seams across segments. I also tested 16 and it wasn't enough, at least for the image I was testing. I also increased the max number of segments to 128 (max size 288x288 each), so its possible to generate up to a 36864x36864 image now, but expect it to take days.

Edit1: Oh yeah, I also updated the installer to use rapka's fork of ESRGAN, so those who installed from before may want to wipe out ESRGAN and reinstall as it comes with several more models. The default path is "C:\SuperScalers", which can now be configured on the installer. You may want to back up the SFTGAN models so they don't need to be redownloaded.

Edit2: A random texture I used to test, from razius's SPM pack. Alpha is correctly applied in both instances.

Spoiler: (Show Spoiler)

Texture: 576x576 - ESRGAN Manga 2304x2304 - SFTGAN 2304x2304

Edit3: Posted an updated script at the top. There is an issue where the "ESRGAN/LR" folder tends to disappear and the image will fail because of it (which I fixed awhile back). Appears the same issue can happen with the "SFTGAN\data\samples" folder. The same fix was applied, which just creates those folder on demand. I also made it so segments will try to be square. If a rectangle texture was upscaled such as 1024x768, segments would end up being 288x160. They will now be carved into 288x288 segments, which means less segments needed and less seamless pixels eating up precious time.

Edit4: Just wanted to post another example. This is just some random mountain image that was on my hard drive. While ESRGAN manga model is great for general artwork, I think SFTGAN has it beat for nature/environments. Also tested SFTGAN pre-upscale with "Cubic" filter and it was a mess. Barely any gains and remained really blurry. It looks like nearest neighbor (point) is the way to go here.

Spoiler: (Show Spoiler)

Texture: 1024x512 - ESRGAN Manga 4096x2048 - SFTGAN 4096x2048

Edit5: Did a few more upscale tests here: https://imgur.com/a/QcwQZQu

[Matrix2525]

Amazing work Bighead! As far as I can tell, all the upscaling features seem to work fine, although I ran into a one-time issue testing the segmentation feature:

Spoiler: (Show Spoiler)

I will say, I "think" it was just an issue on my end. I've been doing a lot of work on my Sonic Adventure 2 Battle texture pack and I think I shredded my VRAM between tons of upscaling and editing in photoshop without a restart over a couple days. After finally restarting, I re-ran the upscale and it worked fine. This was using SFTGAN and a 24 pixel buffer on a 512x512 image. And to get some stats in, with my GTX 1080, the segmented upscale took about ten and a half minutes. Upscaling the same image in one piece took under a minute, so it is indeed quite the difference. Thankfully I don't need to use the segmentation feature on my current project, as the largest textures are 512x512.

Although, I did have a couple textures I upscaled twice and then halved to get 8x before this update hit. So I split a 1024x1024 image in half manually. I found with 8GB of VRAM that I can still use ESRGAN with a 1024x544 image (32px border).

I'll be testing out the Resource Pack features soon as I near the next release of the pack, which should be soon I hope. The size of the pack has increased exponentially with my change in scope from a UI overhaul to covering almost everything. Last release was about 750 textures, and I'm now sitting at 3800+...

[Bighead]

Another one, may as well keep the beta train running.
http://www.mediafire.com/file/3quaricbtuvi4e7/Custom+Texture+Tool+PS+v40.0.b9.zip

I think most issues have been ironed out, but there is still one particular issue that I can't seem to figure out. I'm using ImageMagick to extract the alpha channel from textures. When extracting it from an opaque image that happens to have an alpha channel, the alpha channel extracted appears corrupted. The image should be completely black, but there are white lines throughout.



And when it's applied to the image, it causes the white areas to become transparent on the texture. One obvious way around this is to disable saving the alpha channel, since it's a waste of time to begin with. Another would be to calculate if there are actually any transparent pixels rather than just reading if it has an alpha channel, but that would add a bit more time. But maybe it's worth it to always get a correct image, and not waste time upscaling the alpha channel when it's not even needed.

@Matrix2525: I'm guessing that one segment failed to be upscaled during the creation of that image you linked. I noticed this happening to me if I load a bunch of stuff (like firefox + tons of tabs) while the script is running. Rather than keep going and ending up with a bunk image, I now made it so the image will fail in totality if this happens, with a pop-up message saying something went wrong. I also made a bunch of other small fixes as well. It's now possible to change the upscaling factor of SFTGAN from 2-4, any higher and it completely loses its effectiveness. I think SFTGAN was made for a 4x nearest neighbor upscale, smaller values do work okay though. I could not get the upscale factor working for ESRGAN however, changing the value in the python script just causes it to fail.

Thanks for the comparison of segmented vs. non-segmented, I'm a bit surprised it actually adds that much time, although I'm not really sure if there is anything I can do about it. Did you happen to also upscale the alpha channel, as that would double the time it takes? I imagine at least some of the extra time is a result of all the preparation that is needed (tiling, cropping, cropping some more, etc), but the bulk of it (at least for me) is when the actual filtering takes place.

[Matrix2525]

So, I had both preserve alpha, and tiling checked (though does the tiling option work with ESRGAN/SFTGAN?) in that first test. I unchecked both options and tested again and... 9:43. So, about a forty-five second improvement, but certainly not half the time. Still, it works. Unfortunately, I don't have any other system configurations to test against. And finally, yes, my system was pretty fragmented with lots of stuff open and running when that first test failed.

[SecretCircle]

Not sure if I've set something up incorrectly but the start button is always greyed out, no matter what options I choose. Any idea what a common cause of this could be? Everything in the tool paths menu seems to be set up correctly and both the texture and output paths are correct as well.

[Bighead]

@SecretCircle: The start button should only be grayed out on startup if the script doesn't find ImageMagick. It's the only "required" program, much of the script relies on it in some way.
http://www.imagemagick.org/script/download.php#windows

@Matrix2525: Yes, the segment option works for both ESRGAN and SFTGAN so there is (almost) no limit to how large an image can be created. It is much slower to tile SFTGAN since the image is upscaled 4x before being fed to SFTGAN, so there are 4x as many tiles required.

I did notice somewhere I can save some time. Currently I'm tiling an image in a 3x3 grid and cropping out the center. For large images, this can take minutes. It would be quicker to define the pixels I want to keep, and build a custom image rather than tile it.