Custom Texture Tool PS v50.1

[Bighead]

Without seeing the original I'm not exactly sure what the problem is. I can see that it isn't correct, but I can't see all the ways that it is messed up without some kind of reference point.

If I had to take a shot in the dark, maybe alpha is being lost or handled incorrectly when upscaling. I work around this in CTT-PS when using ESRGAN or SFTGAN with the "Preserve Alpha Channel" option, which extract the alpha channel, upscales it separately, and merges it with the result. The downside is it doubles the processing time.

[Dlon93]

Hello, Bighead,

Thank you for creating that amazing tool for extracting TOCS textures. I've been messing around with it today, and I need your help.

I can extract textures for Cold Steel 1 and 2 without any problems. However, no matter what I try, I simply can't extract textures for Cold Steel 3. I have tried everything. I installed following the steps for your easy method, as well as the manual method, and I always get the same issue. All of the tools are properly connected to their right paths, and I have also installed Python. I tried both the version you provided in your "programs pack", as well as the latest version from Python's website. Nothing works.

If I try to extract individual .pkg textures from "The Legend of Heroes Trails of Cold Steel III\data\asset\D3D11_us", your tool gives me that green, 100% status completion "OK" report, like everything worked just fine. However, when I open the output folder, it's always empty.

When I try to extract the textures from the "assets.pka" file, the tool always gives me the following message (see attachment).

Like I've said, Cold Steel 1 and 2 textures work just fine. But Cold Steel 3 textures just refuse to unpack. If you need me to provide any more info, I will. I would really appreciate any sort of help. I am trying to extract and replace those textures that display character names when they are introduced for the first time. I'm pretty sure those are .pkg files in the "D3D11_us" folder.

In any case, thanks in advance.


Dlon93

[Bighead]

Sorry, I know that frustration all too well. When you try just about everything you can think of and still can't get it to work right (for example, I'm a user of MAME and Retroarch and emulate some obscure systems lol). While I have no idea what the problem is, I don't like it when this script just doesn't work for someone. So, I whipped up a quick version that should hopefully point us in the right direction to what the issue is.

CS1/CS2 don't rely on Python at all, and the code for those games has been tested, tested, rewritten, tested some more, rewritten some more, and then some. So it's no surprise they work without issue.

Before you decide to mod textures in this way, have you tried using "SpecialK" by Kaldaien? There are several caveats to extracting and replacing the textures manually in CS3/CS4.

- You can't repack PKA files. This means that the "assets.pka" file needs to be replaced with all the PKG files extracted from it.
- You can't repack PKG files using LZ4 compression. This means much bigger files.
- You can't replace most UI textures with higher resolution images. They will not scale correctly.

I made a post >here< on the Steam forums that details all the pros and cons. If you are fine with all of this, then read on....

I attempted to replicate your setup by copying "assets.pka" to the desktop. First, download this version:
https://www.mediafire.com/file/2oc2rl28bkndgca/Custom_Texture_Tool_PS_v47.0.b31.d.01.zip/file

Open up the script, enter the "Options" menu, go the "Debug" tab, and show the PS console.

Spoiler: (Show Spoiler)

You may notice a new option to disable online checks. This shouldn't affect your issue, I just put this here because of a failure on my part. My internet has been relatively unstable today, and I realized without a connection, the script just freezes in an infinite loop of retries! Basically, this disables the check to see if all the required dependencies are installed. While the check itself shouldn't require internet, just the download if something is missing, this is yet another issue I will have to look into later (Edit: Found the issue, was trying to update pip every time. Will be fixed/removed next version).

Spoiler: (Show Spoiler)

Now run the option as you would normally, but make sure to extract only a specific file, it doesn't matter which one. The reason being, there is 2 code paths (one for single files, one for all files), and I only modified this one. The console will output a bunch of info, but we're only interested in the section just before it pauses. Whatever is printed out here, please send that to me via screenshot (or you can just copy and paste all the text).

Spoiler: (Show Spoiler)

If it shows some info you don't want to share publicly, you can either filter it out (edit with paint or something) or just send it to me privately via PM, email, whatever. Hopefully this will at least let us know exactly what is failing. If not, I'll try something else.

Somewhat related, I noticed an issue with PKA/PKG validation: the script was validating image files when it should only be validating PKA or PKG files (depending on the operation). This is also fixed.

[Dlon93]

Thank you so much for responding so quickly, and kindly. You could've just said "Don't know what the problems is. Sorry." and be done with it. But you didn't. I appreciate it.

In any case. I definitely want to use this method for modding Cold Steel 3 textures. I'm not a fan of using the Special K method.

I'm pretty sure that I just need a few ".PKG" textures from the Cold Steel 3's "D3D11_us" folder, and I doubt I'll be opening the "assets.pka". Like I said, I simply need to change a few small textures that display character names, for a project I'm working on.
----------------------------------------------------------------------

Now then, I followed your instructions, and it appears that there's a "Scripts" folder missing from the CTTP folder. Python is trying to point to it, but can't find it. I manually checked the location, and that folder (Scripts) is not there.

Inside the "C:\Users\brkov\AppData\Local\CTT-PS Data" folder, I only have:

1) Programs
2) Temp
3) StoredOptions.ini
-------------------------------------------------

I've included a screenshot of the console that you've requested, so please let me know what I should do next.


Thank you!

Dlon93

EDIT: I get the error shown in the screenshot below, both when extracting regular Cold Steel 3 .pkg files from the "D3D11_us" folder, as well as when trying to extract the "assets.pka". In both cases, Python can't find the "Scripts" folder and the "pkg_extract.py".

[Bighead]

Thank you for alerting me of the issue and helping to find the source. Yeah, that's definitely a problem. CTT-PS should automatically create those script as needed. But since it didn't, I implemented a new method of creating them.

Please try this version and use it as you normally would:
https://www.mediafire.com/file/ewidmj9qhihli8q/Custom_Texture_Tool_PS_v47.0.b31.zip/file

I have removed all the logging info, so it's back to it's normal self. I have made a few other changes as well.

- Removed that online verification thing. Now the script will only attempt to update Python stuff if it is actually missing, and it will now prompt the user when no internet connection is found.
- Detection of which game PKG files come from, so no need to select a priority anymore. I found that byte 85 can be used to determine the type of PKG file.
>> 1 = CS1/CS2 (LZ77), 4 = CS3/CS4 (LZ4), and 0 = Uncompressed. The last one is interesting, only a single texture is found uncompressed across all games ("I_SWIM_RENDA.pkg" in CS3).
>> This is much cleaner than the old implementation that just tried to hit every PKG file with the PKG Tool, then if failed, hit them with the Python script.
- PKG file game and compression type is now printed to the texture list.
- Improved error reporting. No more false positives saying files were extracted when they weren't.
- PKG/PKA validation fixed. Was passing texture files, which didn't cause issues but was wrong anyway.
- Dead code removal.

In the worst case scenario where the script doesn't create the "scripts" folder, you can try to manually place them there (although I'm hoping this won't be necessary). I also included the PKG file lists which just go into the root of the data folder, which should also have been automatically created.
https://www.mediafire.com/file/dque9752cmhi26l/CTT-PS_Data.zip/file

[Dlon93]

(07-04-2021, 02:13 AM)Bighead Wrote: Thank you for alerting me of the issue and helping to find the source. Yeah, that's definitely a problem. CTT-PS should automatically create those script as needed. But since it didn't, I implemented a new method of creating them.

Please try this version and use it as you normally would:
https://www.mediafire.com/file/ewidmj9qhihli8q/Custom_Texture_Tool_PS_v47.0.b31.zip/file

Thank you, once again, for a swift and helpful response. The new version you linked above is working as intended. The "Scripts" folder has been created, and the tool is now properly extracting the textures for Cold Steel 3. Awesome! Thank you so much!

I'm gonna go mess around with it now. Can I add you as a friend on STEAM? It might be better to bug you there, in case I run into any more issues, instead of spamming this thread?

In any case, thank you for everything.


Dlon93

[Bighead]

Glad to see that it's working now. And sure thing you can add me on Steam. While I try to avoid non-Dolphin related conversation on this board, it can't be avoided sometimes since this Cold Steel stuff is part of my texture tool. Although, this thread has been rather still lately, so at least it might make for an interesting read to... someone.

Overall, I think it was just a dumb mistake on my part. My guess is the problem was I had to create the folder first before creating the scripts. Usually when creating a file/path with "New-Item", all folders/subfolders are automatically created. But since I was using "Set-Content" for the script files, it most likely expected the folder to already exist.

[Dlon93]

Alright, I added you. Thanks again for your help! This tool is great. It works just fine. I hope we'll figure out the new LZ4 encryption one day, so that you can make it even better. Talk to you later!

[Bighead]

The time has finally come. After over a year of beta versions and many updates, the release of v47.0 is here:
https://www.mediafire.com/file/6x9srnrththpoo6/Custom_Texture_Tool_PS_v47.0.7z/file

The updated programs pack can be found here:
https://www.mediafire.com/file/6guubk0w4se7127/CTT-PS_Programs_(8-2-2021).7z/file

If doing things manually, I highly suggest updating to the latest version of CompressonatorCLI.
https://github.com/GPUOpen-Tools/Compressonator/releases

The full changelog can be found here, which covers everything that I didn't forget to add:
http://bhemuhelp.unaux.com/other/ctt/CTT-PS Changelog.txt

---

So much has been added over time that it would be impossible to cover everything in detail, so I'll just go over the highlights. Most of this stuff won't be new to anyone who has been using the beta versions, which I assume is most people.

High DPI support: The GUI now scales correctly with higher DPI settings. Before this change, Windows auto-scaling was taking over the GUI, which applies a bilinear filter resulting in a blurry GUI. I was not able to figure out how to make the script "DPI Aware", so I created my own method of DPI scaling which works really well. I covered this in-depth in this post here.

CTT-PS Data Folder: As much as I wanted to keep the script self-contained, it became impossible the more complex it became over time. So a centralized location was needed to keep all the external files. The CTT-PS data folder is that location, and can be located to AppData, Documents, root of a HDD, the script folder, or anywhere else the user wants to put it. There is a drawback to a custom location however, future versions may not be able to find it without manually locating it.

Store options in an INI file: The script used to save all of the user's changes to the script file itself. While it is still possible to do it this way, the default behavior is now to save it to an external INI file named StoredOptions.ini, found in the CTT-PS Data Folder. This has the advantage that new versions of the script will no longer need a manual import of all options to retain settings between versions.

Redesigned Options Menu: The "Options" panel has been broken into two categories for "Behavior" and "GUI elements". This offers no advantage other than it's a bit neater to look at, and separates the options on what they do.

Split channels into separate images: Sometimes multiple alpha textures are combined into a single image, using each channel to store the image. It's now possible to split each channel into a separate image, perform some work on them, and recombine them. More info can be found in the posts here.

Create Materials for Ishiiruka: While it's been possible to combine materials into material maps using Ishiiruka Tool for a long time now, there was no way to generate the materials without using a paid program. Thanks to JoeyBallentine and his Material Map Generator based on ESRGAN, we now have a nice and easy way to create materials from color textures.

Rescale Dolphin "Real" Dimensions: The Rescale operation uses the dimensions in the filename of Dolphin textures, while using the real dimensions of other textures (while in Expert Mode). An option has been added to instead use the real dimensions of Dolphin textures when calculating the new dimensions (dimensions * rescale factor). This makes it possible to easily create a half-size pack by setting the Rescale Factor to 0.5, or 50% of original.

Run any CLI Program with CTT-PS: Over time I have had many people ask me to support different programs in this script. And being only one person who also lacks the interest in doing so, it would be too much work to add and test everything suggested. So as a compromise, this new operation allows adding any CLI-based program to the script.

Multiple PNG Optimizers: It started with OptiPNG. But many alternative have emerged over the years, and I have had requests to support their favorite program. So now it's possible to use OptiPNG, OxiPNG, Pingo, and Efficient Compression Tool to optimize PNG files.

Configuration for DDS Programs: It's now possible to tweak the programs that create DDS textures, namely TexConv and Compressonator. While there isn't a huge amount of options to change, some can be useful to create higher quality textures or speed up the operation when creating them. A [Configure Tool] button can be found under DDS Options.

Force Create Mipmaps in Dolphin Mode: This option used to be hidden in Dolphin mode, and now it is exposed. I learned that Dolphin allows using mipmaps for textures that don't have the "_m_" flag which is fantastic, so now it is possible to force generating mipmaps for all textures. This should not be done for all textures though, only environments that may benefit from mipmaps to prevent shimmering.

Process Selected Rewrite: It was relatively slow to add many textures to this window, and that has been greatly improved to the point its nearly as fast as skipping previews altogether. There also used to be a limitation that two textures from different locations could not share the same name, and that has also been fixed. Several other small bugs with this feature were also fixed.

Automate Metroid Prime 2 fonts: Metroid Prime 2 has a lot of duplicate font textures with different colors. And when I say a lot, I mean hundreds of them. Now there is a way to automate the creation of these fonts by supplying a single HD texture for each one, with the slight drawback that the variations must first be dumped. More on that here.

Cold Steel III / Cold Steel IV support: It's now somewhat possible to modify textures in these games, thanks to some python scripts by CeruleanSky. Textures can be extracted from the PKA and PKG files, modified, and put back into the game. There is still many limitations, as there is still no recompressor for LZ4 and the scaling issues from CS1/CS2 have made a return. More on that can be found here.

Skip counting textures: Ever try to convert a pack with thousands, even tens of thousand of textures, and it hangs for several seconds, even over a minute when the operation is started? This is due to the script counting and filtering out textures that won't be processed (like mipmaps which count as part of the main texture). Using a debug option, it's now possible to skip this count completely, but this also breaks the progress bar. Hopefully I'll be able to speed it up in a future version.

---

That's it for now. The reason I didn't release this sooner is because of the -Peformance command bug in Compressonator. I said I would release an update when it was fixed, and it took much longer to see a release with that fix than I thought it would. From this point on, I'll definitely be updating the "main version" with much smaller changes.

[djneo]

great work as always, your work is highly appreciated!

Thanks!