Custom Texture Tool PS v50.1

[Bighead]

@MamiyaOtaru: The upscale rewrite took much longer than expected, but hopefully this should fix everything.
http://www.mediafire.com/file/38ybwvdagjjmk6i/Custom+Texture+Tool+PS+v40.1.zip

I have not yet addressed some of the other issues mentioned, I plan to release a followup sometime soon.

[Malkierian]

So, to follow up with my Compressonator issues, I went ahead and completed the arduous process of setting up my machine to be able to build the CLI myself (it took me 4-5 hours over the course of many days because of various downloads I needed to do [saying 15 would not be much of an exaggeration]; their "build from source" guide was all over the freaking place). Guess what? It works! Not only that, but the one I built didn't require admin rights! So something was obviously wrong with their latest release. But, I don't think I'm going to get any help from them, in my GitHub issue, they seemed less than enthusiastic about even exploring possibilities with me. Is it possible you might be able to just distribute the tool with .exe's you know are working with your tool? Or perhaps, in the case of Compressonator, switching over to using the libs internally instead of the external CLI?

[Bighead]

Sorry I still haven't dived into the Compressonator issue myself and did any testing as its been a rough couple of weeks. I struggled to get the last version out because upscaling filters had far more issues than what was initially reported. As for Compressonator, it turns out I have been using v3.0.3707.0 all along, which is a version old. I installed the latest CLI version, I'm not sure why they divided the GUI and CLI into separate installers. A quick test and its not working at all with the script, I will assume for now the issue is exactly as you mention.

Is it possible you might be able to just distribute the tool with .exe's you know are working with your tool?

Yes I could do this, and I've considered it for years ever since it was a batch script. The reason I haven't is because I didn't want to ship old exe's and maintain them myself, some programs are constantly updated (ImageMagick for example). This puts the responsibility on the user to get the latest versions of the programs, and updates to a program can come after I make a script release. Obviously this comes with its own issues, as not all updates are always "good" and may have regressions. But most of the time, we can usually assume that updates are good.

Or perhaps, in the case of Compressonator, switching over to using the libs internally instead of the external CLI?

This is most likely beyond me at the moment. While I probably could figure it out, I know next to nothing about how to do something like this and I currently don't have the time to put in the research. I am not a programmer in the traditional sense of the word, and to be honest, I'm terrible at coding and a slow learner. While this script could be classified as a "program", it's still just a script, and the only reason I can understand PowerShell to the extent that I do is because I used to mod games and write scripts that used similar syntax when I was younger (WarCraft III, StarCraft 2). It's only come this far out of dedication and 100's of Google searches.

With all that said, there is definitely something up with the latest Compressonator. I have all errors hidden in the script at the moment, so I don't know if the issues I'm having are exactly as you mentioned. So for now I think sticking with 3.0.3707.0 is probably the ideal solution for the common user. When I do end up releasing v40.2, I think maybe I'll make a separate download for an "addon pack" that has all the programs needed in a folder that can just be dropped into the same folder as the script, and everything just works. That way the preferred method would still be to download the programs individually, but for those who don't want to deal with that can take the easy road.

[collige]

(04-24-2019, 01:36 PM)Bighead Wrote: Hello collige/rapka, now I can thank you personally for your ESRGAN fork that includes the models as it simplifies the installation process, and your scripts as it was a big inspiration to mine when adding ESRGAN/SFTGAN. I didn't understand your code since I'm awful at all things programming and only understand PS (and some C# since it can be translated). But, the concepts themselves such as extracting/reapplying alpha and excluding textures with ignore.txt and excluding dimensions like "320x240" led to the exclusions list in my own script.

I do have one question, in your instructions you mention this: "These upscaled images are then gamma corrected to account for noise introduced by the algorithm and are then merged back into the main image." I'm not sure what you mean by gamma corrected. To get the alpha channel I'm using ImageMagick to extract it, then its upscaled, then reintroduced using ImageMagick. And I don't think it's doing anything with the gamma that maybe I should be doing? So far the results have been good, but gamma correction can make them better?

(Sorry for the extremely delayed reply, life happened and I kinda forgot I signed up here)


The gamma correction was a hack because I was under the impression that the graphics library I'm using didn't have another way to easily affect image brightness, but that's not the case. Amongst other things, I'm revamping my script to take in options to determine exactly how channels are manipulated. The downside is that it unfortunately means that there's probably not going to be an easy one-click solution for throwing ESRGAN at Dolphin games since various games handles transparency differently on top of the FMV dump issue. For example:

In Kirby 64, pixels that are partially opaque are rendered as fully transparent
In Tony Hawk's Pro Skater 3, pixels that are partially opaque are rendered as fully opaque. This was the original issue that led me down this rabbit hole
In Need For Speed Underground, the game appear to use some kind of color-to-alpha algorithm as some opaque images with black backgrounds are rendered without backgrounds

Because of these variations in game implementations and because ESRGAN generally introduces noise into whatever image it upscales, you end up with visual glitches in the above examples if you don't do any correction to make sure the right pixels are hitting either 0 or 255.

On an unrelated note, my biggest annoyance in getting into this so far has been dealing with procedurally generated textures that make it borderline impossible to work with some games (N64 games in particular) that aren't able to be sorted just based on their resoultion. Have you had any success in finding a way to identify them?

[Bighead]

Thanks that was a great explanation. I fully understand what you mean as some textures can behave the same way in Dolphin for certain games. Paper Mario: The Thousand Year Door renders partially transparent pixels on "most" sprites as fully opaque, IIRC. As for N64 emulation, I never really worked with custom textures. Unfortunately there just isn't enough information in the dumped name, and a dump is always PNG, so I have no idea how you would sort them.

[Bighead]

(05-09-2019, 05:49 AM)Malkierian Wrote: Ah, here's another one for you: there's actually no way to make a custom output folder name.  You can't pick the folder "~CTT_Generated" goes in, let alone change the name of that folder or the subfolder ("DDS_BC3").  Any changes I make in "Output Folder" text box just revert to the default within the texture folder parent.  Is there any, safe, way you could allow those changes?  Like, right now, I'm experimenting with different mipmap depths, so it'd be nice to be able to rename something to include the number of mipmap levels generated.  If not, that's fine, I'll do it manually, just one of those "would-be-nice" things.

Not sure if you are still paying attention to this thread, but I forgot to respond to this and wanted to explain how this works, and why it works the way it does.

How it works:
The output folder must exist to select it. So if you wanted to output to desktop, you would need a folder on the desktop. For example, create a folder on desktop named anything like "OutputFolder". You can then select this folder as the output path, or drag and drop it into the script's output path. This will create an output folder named "~CTT_Generated" within this folder when you create textures. The full path should be:

C:\Users\Username\Desktop\OutputFolder\~CTT_Generated

Every operation creates a unique folder name within this directory with the textures. You can automatically rename this folder which should get what you want, but only when using the "Convert" or "Rescale" options. You will see a checkbox for "Auto-Rename Output". Check this, and enter what you want for the output folder name (in your example, DDS_BC3). When converting is done, the output folder will be named to what you want, instead of something like  "ConvertedTextures(PNG)". So the full output would be...

C:\Users\Username\Desktop\OutputFolder\~CTT_Generated\DDS_BC3\<textures here>

Why it works this way:
The main reason is because of how all my operations are coded to output textures. Much of the core design of this script was based around my original batch script version. That version, as well as my early PowerShell versions, needed to be copied directly into the root of the texture pack. There was no option to select the "Texture Path" until a GUI was created. This means the output path was always output to the same path as the textures, hence why the script still adds the "~" character to skip this path. I kept it around because it's safe, the output path may often still end up in the texture path. Without some way to say "skip this path" the script can enter an infinite loop. It processes all textures, then the ones in the output folder, then the ones in the output-output folder, then the ones in the output-output-output folder, and so on...

So in short, you can customize the output folder, but only with Convert and Rescale, and they will always go to ~CTT_Generated folder because it's safe from an infinite loop in the case the output is within the input. This design also has the benefit of allowing to add "~" to any folder to skip it for a session. I could add the "Auto-Rename" to other operations, but I didn't really see a reason to since Convert/Rescale are the ones mostly used, and the only ones that should be used to create a full pack. The main idea behind this option was to rename the output to the Game ID.

[Malkierian]

I appreciate the extended explanation, but I was totally fine with just labeling my folders with the tilde to keep them from being processed. It did what I needed.

I had a little bug to report, as well, regarding removing the transparency from color channels for my MPT text textures. When I select Modify Textures In-Place, it doesn't seem to do anything. Doesn't replace what's there, doesn't make anything new. If I don't have that checked, it makes the modified texture in the ~CTT_Generated folder as normal.

[Admentus]

Suggestion for improvement . I don't recall if I suggested this before, if so, sorry.

That processed textures that were being dragged into the Custom Texture Tool can be outputted in the same folder as where it was dragged from, with the option to overwrite the original file or to have it as a copy. Even in the case of converting from PNG to DDS with the option to remove the original PNG files (and vice versa).


--- Example ---
The Custom Texture Tool is located and ran from: G:\Emulation HD Textures\Custom Texture Tool PS
Live Texture Working Folder is located and ran from: F:\Emulation\Wii\Settings\Load\Textures\PZL\..
So dragging textures from the folder for processing: ..\PZL\Areas\Great Bay into the Custom Texture Tool causes the processed textures to appear in ..\PZL\Areas\Great Bay as well and not into G:\Emulation HD\Textures\Custom Texture Tool PS\~CTT_Generated\..


I probably missed the point from your explanation you posted above, where it seems to be impossible?

[Bighead]

So to put it really simply, you want an "in-place" option for converting/rescaling.
If the output format doesn't match the input format, another option to delete the original or keep them both.
And technically, output path = input path.

Anything is doable, I'll have to think on it for a bit of how it could be implemented. I have never wanted an option like this because of how destructive it can be, but I can understand its uses. I never diverted too much from the "~CTT_Generated" folder because it is safe, someone cannot end up accidentally destroying their pack which could ultimately be my fault for not presenting it in a way that the user understood before activating.

The operations that do have in-place checkboxes are "mostly" safe as everything can be reversed. Material Maps I don't like the in-place but Ishiiruka Tool has it, OptiPNG should be safe to replace directly, mipmaps in-place can easily be reversed, extracted DDS mipmaps can be restitched, and alpha manipulation is so niche that nobody should ever be using it on a full pack of textures. Any "damage" caused by these options can be reversed. Converting all textures from PNG to DDS is permanent quality loss, rescaling textures permanently replaces the original, having both PNG/DDS in the same folder is a mess someone might not know how to easily clean. Any of these could lead to "Bighead your tool destroyed my only copy of my textures!!".

I do have interest in making a global "In-Place" option possible since it would be useful, so it's now on my to do list. I just don't know when I will get to it, it wouldn't be in the next version but maybe the one after that. I am planning on making a small update soon that is kind of like a "cleanup" for all kinds of tiny issues recently brought up, also adding one more alpha manipulation option, as well as rewriting several of the tooltips to be more concise and consistent. I want to get this stuff out of the way before tackling a big one like figuring out the UI stuff and rewiring how my output works.

[Bighead]

@Admentus: I think I was totally getting what you were suggesting wrong. I think you were talking about the "Process Selected" menu rather than all Operations. Let me know if this is what you were really asking for.

Spoiler: (Show Spoiler)

I was thinking of adding something similar that would work without "Process Selected" menu, across all operations, which is what I was talking about in my above post. But I still don't know if it's such a good idea, it may be safest to isolate the functionality to here. I don't mind it being here since this option is already niche. Only UI stuff is done here, and I've been procrastinating with some other stuff, so it will still be bit longer before I get something out that is ready.