Custom Texture Tool PS v50.1

[Bighead]

That error is a new one to me. I'm not exactly sure why that crash would happen, but I probably know what is causing it. There were a few C# functions added in v49.0 that allows me to create rich textboxes with text justification and transparent labels. But I've never seen them cause problems before. Windows 10 should come with PowerShell v5 and .NET Framework v4.8 should be included in some Windows 10 update. This is everything CTT-PS Needs to run correctly that I'm aware of. So my guess would be that there may be some issue with your .NET files or installation.

[Ven0m0]

(08-13-2022, 02:28 AM)Bighead Wrote: That error is a new one to me. I'm not exactly sure why that crash would happen, but I probably know what is causing it. There were a few C# functions added in v49.0 that allows me to create rich textboxes with text justification and transparent labels. But I've never seen them cause problems before. Windows 10 should come with PowerShell v5 and .NET Framework v4.8 should be included in some Windows 10 update. This is everything CTT-PS Needs to run correctly that I'm aware of. So my guess would be that there may be some issue with your .NET files or installation.

I could try to reinstall powershell v5. I've already reinstalled .net.

[Bighead]

I was gonna make a post on Venomalia's thread but didn't want to hijack it so I posted here instead.

(08-10-2022, 04:38 PM)Venomalia Wrote: the Custom Texture Tool from Bighead could actually do that as well once it supports the TextureExtraction Tool.

I added preliminary support for Dolphin TextureExtraction Tool, the operation in CTT-PS is called Convert Dolphin Textures From Extracted Disc.
https://www.mediafire.com/file/98ylaboljjq44th/Custom_Texture_Tool_PS_v50.2.b1.7z/file

Spoiler: (Show Spoiler)

It can be added to the script via the operation options or the Options/Tool Paths menu, although the former method may be removed eventually if the TextureExtraction Tool has many options to configure. Only supports the "extract" command so far, as well as mipmaps, raw images, and force unknown file formats. I'll probably rename the operation to something like "Run Texture Extraction Tool on Extracted Disc" (or something) when it does more than extract textures. Although, I wonder if the other things are all that useful for the average user like "unpack" and "cut" In fact, I'm not quite sure how cut works or exactly what it does.

One caveat is that CTT-PS was designed with an internal loop that sends files to external programs one by one, and the extraction tool takes a directory and also has its own internal loop. This makes it impossible to do some things like printing files as they are processed in a reasonable way. I want the user to know something is happening, so I do display a message saying its working in the background, the progress bar is set to a repeating marquee style, and the number of textures is updated every 20 seconds; it would be slow to update it constantly (especially on slow HDDs).

Spoiler: (Show Spoiler)

I threw this together over the weekend so I haven't tested it much but it seems to work. Open to suggestions and alterations as well. Also, the "Cancel" button doesn't work yet because I forgot to try to get it working somehow . When pressed the GUI will just be stuck in limbo until it's finished. Since I can't just break the master loop, I'll probably have to do a task kill or something.

[Venomalia]

that's pretty cool!

(08-23-2022, 05:53 PM)Bighead Wrote: I'm not quite sure how cut works or exactly what it does.

The Cut function cuts data into their individual blocks, as a pattern you can specify one or more file identifiers.
e.g. the Tales of Symphonia games consist of .cab (MSCF) files which are composed of several .cab files. if you try to unpack them with a tool like 7-zip only the first block will be unpacked.
hardly anyone will be able to do anything with this function, but if something like this is needed, it is incredibly practical.
internally, it is mainly used as a component of the "Force" function.

(08-23-2022, 05:53 PM)Bighead Wrote: One caveat is that CTT-PS was designed with an internal loop that sends files to external programs one by one, and the extraction tool takes a directory and also has its own internal loop. This makes it impossible to do some things like printing files as they are processed in a reasonable way. I want the user to know something is happening, so I do display a message saying its working in the background, the progress bar is set to a repeating marquee style, and the number of textures is updated every 20 seconds; it would be slow to update it constantly (especially on slow HDDs).

All important processes of the extraction tool are documented live in the DolphinTextureExtraction tool.log, maybe you can just check them.
I would also be willing to create a separate log file documenting only the textures and their properties, if there is a format that is beneficial to you let me know,
some properties like the WrapMode and the FilterMode are very pratic when you want to upscale the textures or to create dds files.

you should also be able to exit the tool via Ctrl+c.

(08-23-2022, 05:53 PM)Bighead Wrote: I threw this together over the weekend so I haven't tested it much but it seems to work.

really great work, I did not expect it to come so quickly.

[Kupo]

(08-23-2022, 05:53 PM)Bighead Wrote: I want the user to know something is happening, so I do display a message saying its working in the background, the progress bar is set to a repeating marquee style, and the number of textures is updated every 20 seconds; it would be slow to update it constantly (especially on slow HDDs).

Not sure if it's related but I remember that the "elapsed time" that CTT shows was always wrong for me. If it let's say spent 2 hours for an operation it only showed like half an hour for example. The seconds elapsed very slowly during the process.

[Bighead]

(08-23-2022, 07:58 PM)Venomalia Wrote: All important processes of the extraction tool are documented live in the DolphinTextureExtraction tool.log, maybe you can just check them.

There probably isn't a simple way to actually update the texture list in CTT-PS unless the extraction tool was able to take a file input instead of a folder input, that way I could pipe the files into the program one by one each "master loop" iteration. Another possibility is if the extraction tool simply output the texture names (or some information) to host/console or some error variable, every loop iteration I could just read what the program sent and update the list with whatever was output. I'm far from a programmer, so bear with me on this. An example is ESRGAN, it runs out of VRAM if an image is too large in size. It sends an error message and in PowerShell I can read that message referencing $Error[0], although it doesn't need to be $Error[0] it just needs something sent to the console (example here). All programs in CTT-PS are ran in a separate PowerShell instance to simulate multi-threading in a task called a "Job". When using Write-Output $Error[0], this is logged in the current job, and when using Recieve-Job in the main instance (example here), I can grab whatever was output from the program from the job instance (I also use this method on other programs that output info to the host to help debug issues). This is a convoluted way of doing it, but it is an avenue that wouldn't require a lot of resources. An example of a lot of resources would be scanning the output folder every so often, seeing what textures exist, comparing the results, printing what is new, but this would be extremely slow. Honestly though I'm content with how it is now, it works, and that's all that matters. No need to start altering your code to fit my awful programming models.

(08-23-2022, 08:58 PM)Kupo Wrote: Not sure if it's related but I remember that the "elapsed time" that CTT shows was always wrong for me. If it let's say spent 2 hours for an operation it only showed like half an hour for example. The seconds elapsed very slowly during the process.

Yeah I've always know the timer was inaccurate but I didn't know it was that bad. It really depends on the operation, most of them aren't affected, some of them the issue is more severe. The problem is anything that causes the GUI to temporarily freeze, also freezes the timer. Usually the freezes are only a few milliseconds in length but they can add up. I'm not sure if I can actually fix it, but it might be possible to reduce the issue somewhat. What operation is the worst offender for you? Specific ones or all of them?

[Kupo]

(08-26-2022, 06:20 AM)Bighead Wrote: What operation is the worst offender for you? Specific ones or all of them?

I mainly use the tool for generating mip maps and converting to dds (bc7). In the NfS games I had this folder for wall and ground textures which had really big textures and where I had very long sessions à 10 hours or so alone for dds conversion. The folder had like 1.000 base textures and another 9.000 mip maps. In a first step I used to generate mip maps for the pngs and in a second stop I converted the textures to dds (also with mip map generation). When I dropped png support for my texture packs I skipped the first step. Later I also reduced the mip map level from unlimited to 2 or 3.
Not sure if the big amount of textures was an issue or if the reduction of the mip map level had such a big impact but the time that I needed after that reduced drastically to something like 2 hours I guess. And in these 10 hour sessions the elapsed time was very wrong. It's like this whole demanding process slowed down the clock aswell 

[Venomalia]

(08-26-2022, 06:20 AM)Bighead Wrote: There probably isn't a simple way to actually update the texture list in CTT-PS unless the extraction tool was able to take a file input instead of a folder input, that way I could pipe the files into the program one by one each "master loop" iteration.

this would be technically possible but would take longer because the tool would have to recognize the already extracted data each time and many processes that would speed up the whole process would not work. In addition, there are also games that have only one large archive, which brings us back to the same problem. :/

(08-26-2022, 06:20 AM)Bighead Wrote: Another possibility is if the extraction tool simply output the texture names (or some information) to host/console or some error variable, every loop iteration I could just read what the program sent and update the list with whatever was output.

yes, currently the console window is quite empty, it's no problem to display the textures and their properties or at least to add the option to do so.

(08-26-2022, 06:20 AM)Bighead Wrote: I'm far from a programmer, so bear with me on this. An example is ESRGAN, it runs out of VRAM if an image is too large in size. It sends an error message and in PowerShell I can read that message referencing $Error[0], although it doesn't need to be $Error[0] it just needs something sent to the console (example here). All programs in CTT-PS are ran in a separate PowerShell instance to simulate multi-threading in a task called a "Job". When using Write-Output $Error[0], this is logged in the current job, and when using Recieve-Job in the main instance (example here), I can grab whatever was output from the program from the job instance (I also use this method on other programs that output info to the host to help debug issues).

I must admit that I have never worked with PS and have no idea about its possibilities. If the extraction tool has a problem, it logs this and the next possible step is performed. These are already output in the console but are probably not important for you. for example, If a file in an archive is read with an error, all other files in the archive will continue to be processed normally.

[Bighead]

New beta version with better support for Venomalia's TextureExtraction Tool:
https://www.mediafire.com/file/gcn0pdv21e1bmzs/Custom_Texture_Tool_PS_v50.2.b2.7z/file

I've been trying to get this out for awhile now, but free time for me is a luxury these days. I don't have much time during the week to work on things, so I was really looking forward to my weekend, far more than most weekends because I had nothing to do and nobody to bother me. Then last Wednesday night I started to feel a little ill. Then Thursday I felt like death, body aches, worst headache I ever had - massive pain behind my eyes, sore throat, which persisted until Sunday afternoon. Turns out I finally caught the big C that's been going around since 2019. No sympathy desired, despite a few days of hell on earth, I consider myself extremely lucky that it didn't hit me hard at all compared to what it does to some. For example, I didn't even have a fever the entire time. And after 5 days I'm already feeling much much better, just tired and sore. I'll probably be down for a little bit longer, so now is as good a time as any to upload this.

Something to note: The two tools working together is mostly just a "nice to have" feature. Running the ExtractionTool through CTT-PS is slower than running it directly. This will always be the case, as the script is always going to add some amount of overhead. It's currently not too bad, but it could be better. There are more things I wanted to try to reduce that, maybe cut down on the many properties extracted and/or optimize the loop further, but for now I know it works well enough to be usable. Could be a few issues I missed too. When I'm feeling better I'll see if I can get the processing time down further, even if it means taking out the various texture properties it displays. Or if people find the properties as a desirable trade-off then I'm also fine just leaving it as it is if I can't make it any faster.

[Blushock]

Sorry if this has been asked before, but can this dump textures from games, or do you have to do that from Dolphin?