Custom Texture Tool PS v50.1

[Ivan89el]

(03-13-2019, 10:09 PM)Bighead Wrote: I have a test version ready for ESRGAN/SFTGAN.
http://www.mediafire.com/file/dkec4ti1awpcf18/Custom+Texture+Tool+PS+v40.0.b1.zip

As far as I can tell, ESRGAN works fine, but SFTGAN is failing on a lot of images and I can't exactly figure out why. I'll go more into that later, but first I want to explain the set up process.

---

IF YOU HAVE NOTHING REQUIRED INSTALLED

This is how to set everything up the easy way: Go to Options >> Operations Tab >> Automatic Setup button on the ESRGAN/SFTGAN section. There are 6 steps, just make your way through them and follow them closely.

---

IF YOU HAVE EVERYTHING REQUIRED INSTALLED

If you already have Python, Cuda, and the filters already up and running, you can simply link them to the script on the "Tool Paths" menu. If Python v3.6 is already installed, it should pull the install path from the registry.

---

AFTER EVERYTHING IS SET UP

If the models don't show up after adding them, and you are sure all the paths are correct, close the options menu and reopen them and they should appear in the lists. After everything has been set up, ESRGAN and SFTGAN should appear in the upscaling filters list.

---

CURRENT ISSUES

ESRGAN works fine, but I am having issues with SFTGAN. I learned most of everything from this blog post here. It was mentioned that SFTGAN requires upscaling the image 4x before running it, but when I attempt this, I get an error in the segmentation test that makes the whole thing break down.



The segmentation test is required. SFTGAN requires running 2 scripts: the segmentation test, and then the SFTGAN test. The segmentation test breaks the image down into multiple images, then the SFTGAN test does the magic. Unfortunately, something is failing on the segmentation test when I upscale images, and often when I don't. I won't have time to try to figure this out for a few days, so hopefully someone who has experience with SFTGAN can point me in the right direction.

The code that prepares SFTGAN can be found on line 2100. On lines 2113 and 2114 is where the upscaling takes place, simply changing the upscale factor for width and height to 1x (for example, $RawImageData.Width * 1) makes many images work, but they don't get upscaled so it defeats the purpose...

Spoiler: (Show Spoiler)

And the actual "job" to do the upscaling can be found on line 2052, but I don't think it is at fault here as it is just calling the program in a separate PS instance.

Spoiler: (Show Spoiler)

SFTGAN is currently coded to write to the console with Out-Host, so if the PowerShell console is enabled all the errors will be visible.

Another issue is that the progress bars don't fill all the way up during the automatic installation. I have no idea why, but it does not affect the installation in any way.

That's great! Now I will try.

[Admentus]

Nice work on supporting ESRGRAN.

I found this issue when going through step 3/6: Update Python in the Automatic Setup

Code:

You cannot call a method on a null-valued expression.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b01.ps1:19138 char:3
+   $PythonInstall = $Python.Replace('\python.exe','')
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull

Updating Python package installer...

The expression after '&' in a pipeline element produced an object that was not valid. It must result in a command name, a script block, or a CommandInfo object.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b01.ps1:19146 char:5
+   & $Python '-m' 'pip' 'install' '--upgrade' 'pip' | Out-Host
+     ~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : BadExpression


Downloading and installing PyTorch...

& : The term '\Scripts\pip3.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b01.ps1:19152 char:5
+   & $PythonPip3 'install' 'https://download.pytorch.org/whl/cu100/tor ...
+     ~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (\Scripts\pip3.exe:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

& : The term '\Scripts\pip3.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b01.ps1:19153 char:5
+   & $PythonPip3 'install' 'torchvision' | Out-Host
+     ~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (\Scripts\pip3.exe:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException


Downloading and installing OpenCV...

& : The term '\Scripts\pip3.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b01.ps1:19159 char:5
+   & $PythonPip3 'install' 'numpy' 'opencv-python' | Out-Host
+     ~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (\Scripts\pip3.exe:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException


Finished!

[Bighead]

That means it failed to pull python from the registry. I'm going to make a guess that it doesn't install to the same location based on install preferences.
I'm guessing it went to HKEY_LOCAL_MACHINE instead of HKEY_CURRENT_USER. For me this entry is located in: HKCU:\Software\Python\PythonCore\3.6\InstallPath

You may be able to select the python path manually in the paths menu for now, but I'm not 100% sure it will save to the script just yet when closing. I'll attempt to make a more robust search and fix the saving issue later. Right now it's only searching that one specific entry, so this check would also fail when starting if Python v3.7 is installed. I'll update this post (or repost) sometime soon.

[Admentus]

The script is having issues setting the path to the Python executable and is not saved on exiting the tool.

Re-opening the tool always causes the Python Executable path to be empty.

(03-14-2019, 12:50 AM)Bighead Wrote: You may be able to select the python path manually in the paths menu for now, but I'm not 100% sure it will save to the script just yet when closing.

It is exactly as you predicted .

[Bighead]

This should hopefully fix Step 3 which is necessary to succeed (you can skip step 1 and 2 if you already installed Python/CUDA). It will now search for Python 3.6 - 3.9, which 3.8+ doesn't even exist yet, in both HKLM and HKCU.

Selecting the path should now also save it to the script if it has changed since the script started.
http://www.mediafire.com/file/nkbb4pm442ao68f/Custom+Texture+Tool+PS+v40.0.b3.zip

My current solutions are not ideal, but hopefully they can at least get it working for now.

Edit: Updated to beta 3 since I forgot to search Wow6432Node, which is a possible location it may end up...

[Admentus]

Still the exact same error shown in the Debug Console.

Python 3 however is now correctly set and saved in the Tool Paths Menu.

[Bighead]

Hmm, well that's an issue. I suppose I could have users manually select the path during that step, but I would prefer to automate it if I can.

Can you open regedit and find the python key? I'm curious where it's ending up...

Example: (Show Spoiler)

[Admentus]

Actually, I solved it by launching the Python CMD (Python.exe, in the installation folder root) once.

It seems it was not located in the registry, by launching it once it showed up in the registry at the usual place: Computer\HKEY_CURRENT_USER\Software\Python\PythonCore\3.6\InstallPath

At least step 3 seems to continue now.

I think I am good to go now. Let's test...

Following error occurred, while running the tool in export mode to convert a .JPG image... It should probably test it properly on a .PNG image.

Code:

Cannot move item because the item at 'C:\SuperScalers\ESRGAN\results\baseimage_rlt.png' does not exist.
+ CategoryInfo          : InvalidOperation: (:) [Move-Item], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.MoveItemCommand
+ PSComputerName        : localhost

--- Job Loop Ended
--- Job Loop Started (ImageMagick - Convert)
<< Job Loop Iteration >>
<< Job Loop Iteration >>
<< Job Loop Iteration >>
magick.exe: unable to open image 'D:\Programs\Textures\Temp\CTT-PS_Temp\TempImages\filtered.png': No such file or directory @ error/blob.c/OpenBlob/3485.
+ CategoryInfo          : NotSpecified: (magick.exe: una.../OpenBlob/3485.:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
+ PSComputerName        : localhost

--- Job Loop Ended
Move-Item : Cannot move item because the item at 'D:\Programs\Textures\Temp\CTT-PS_Temp\TempImages\final.png' does not exist.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b03.ps1:7054 char:3
+   Move-Item -LiteralPath $FinishedImage -Destination $OutputFile -For ...
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [Move-Item], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.MoveItemCommand

Get-Item : Cannot find path 'D:\Programs\Textures\Temp\CTT-PS_Temp\TempFiltered\Link's House - Up.png' because it does not exist.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b03.ps1:7074 char:19
+   $FilteredItem = Get-Item -LiteralPath $TempFilterFile
+                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (D:\Programs\Tex... House - Up.png:String) [Get-Item], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand

Get-Item : Cannot find path 'D:\Programs\Textures\Temp\CTT-PS_Temp\TempFiltered\Link's House - Up.png' because it does not exist.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b03.ps1:2873 char:26
+   $ImageItem           = Get-Item -LiteralPath $ImageFile
+                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (D:\Programs\Tex... House - Up.png:String) [Get-Item], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand

Exception calling "ReadAllBytes" with "1" argument(s): "Det gick inte att hitta filen D:\Programs\Textures\Temp\CTT-PS_Temp\TempFiltered\Link's House - Up.png."
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b03.ps1:2878 char:3
+   $ByteArray = [IO.File]::ReadAllBytes($ImageFile)
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : FileNotFoundException

You cannot call a method on a null-valued expression.
At F:\Emulation\Wii\Custom Texture Tool PS\Custom Texture Tool PS v40.0.b03.ps1:2888 char:3
+   $ImageInfo.Dimensions = $ImageInfo.Width.ToString() + 'x' + $ImageI ...
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull

Using it on a .PNG image won't work either. Same issue.

EDIT:
Ohh... I got it too work... The resolution of the image was too large. Tested it on a different image, this time 400x300.

Ok, the tool works fine. Easy way to get going with ESRGAN. Should hopefully make it much easier for me to upscale those infamous Ocarina of Time backgrounds.

[Bighead]

Hmm.. the path to python not being entered into the registry until launched is a real pain. I wonder why that is happening, it creates the entry immediately on my end. If this is the case, I would need the path to first launch it, and I can't launch it until I have the path. D:

I guess an alternative would be to manually search the hard drive if the registry search yields no result. That will end up being slower, but it can be sped up by looking in known locations first ("C:\users\username\appdata\local\programs\python" or "C:\python"). And if it fails, I guess the only option is to have the user manually input the path during that step.

Also, I didn't do much testing so I didn't realize ESRGAN can fail with large images. I think that's the same issue I've been having with SFTGAN, resolution being too large (or something). I wonder what the limitations are on these upscalers, as the only info I've really been able to find was "look how fancy these look!".

I have output from ESRGAN blocked because I thought it was working correctly, so the errors won't help me too much (although, I don't think the ESRGAN specific errors will help me too much either ). Next version I'll have them output to the console since I'll have to do some more testing.

---

Edit: This version should up the success rate on finding python when doing the auto-setup (and in general, the same search is done on boot).
http://www.mediafire.com/file/lvg688uyt4u384s/Custom+Texture+Tool+PS+v40.0.b4.zip

If the registry search fails, it will also look in four places for a "Python" folder (so it doesn't scan the entire hard drive). It doesn't do directory recursion or it would be horribly slow, so if it's in a sub-folder, it will fail ("C:\Junk\Python" for example).

- The "Appdata\Programs" folder (default location).
- The root of the primary hard drive (usually "C:\").
- The "Program Files" folder.
- The "Program Files (x86)" folder, although the 64-bit version should be installed, its possible to force installing it here.

If all that fails, it will ask the user to select the path to "python.exe". And if THAT fails (which is only possible through user error), then it will show a message saying ESRGAN and SFTGAN aren't going to work.

Also, ESRGAN will now output any error messages. One of them can be ignored, something about a python function being deprecated.

[Bighead]

I decided to try to figure out why ESRGAN/SFTGAN is failing on larger images. After doing some digging, it seems that both of them consume a ton of VRAM, and if it runs out then the upscale will fail. I'm on a GTX 970 which has 4GB, and I'm having trouble even doing small images. For example, I tried to use it on a 354x500 image, and I get the error message:

RuntimeError: CUDA out of memory. Tried to allocate 1.52 GiB (GPU 0; 4.00 GiB total capacity; 1.59 GiB already allocated; 1.42 GiB free; 2.18 MiB cached)

Obviously these numbers don't add up. Further looking into it seems the feedback from ESRGAN is not entirely accurate.
https://www.reddit.com/r/GameUpscale/comments/avpir1/calculating_required_vram_for_esrgan/

Using the image from before, dividing the dimensions by 2 (177x250) and it succeeds.

I do have an idea that might just be crazy enough to work. If a large image is detected, splice it up and run ESRGAN on each segment individually then reassemble it all at the end, similar to how the "seamless method" and "combine multiple textures" would work if they were one option. Like the seamless method, this brings with it a challenge: each segment can't just be upscaled directly, it starts to get complicated because some overhang would be necessary to make each segment seamless with the next. That overhang will then need to be cropped before reassembly.

I'm definitely not a math genius, so while I know what will have to go into this, it may take me some time to implement correctly. It may be easiest to cut images into predetermined sizes (something like 128x128 - 134x134 with 6 pixels of overhang). The supplied models do a flat 4x upscale, so every segment would end up as 536x536, subtract 24 pixels and we get the 512x512 segment. But to further complicate it, the side segments will only have 3 sides that need neighboring pixels, and corners will only need 2 sides. The math behind it all is pretty basic, but I still tend to struggle with it. The seamless method took me some time to get right, and this is like that amplified to 11.

Another thing I may not be factoring is how it works internally. Does ESRGAN and SFTGAN need the whole image to work as intended? This is something that I will obviously need to test before going through all of this. All in all, hopefully it will be possible, as it would be nice to have an implementation of ESRGAN/SFTGAN without its current limitations. I do expect it to be ungodly slow on large images, as upscaling a single image such as 1024x1024, would actually be upscaling 64 individual images if using chunks of 128x128.