Custom Texture Tool PS v50.1

[Bighead]

By now I am convinced the issue is with linux Dolphin and prefetching enabled. I have a few possible theories on what can trigger it:

- Texture Pack RAM consumption > 5-6 GB (not sure on exact value)
- Single texture RAM consumption > ???
- Texture dimension > some value (2048, 4096, 8192?)
- Possibly only specific to Vulkan backend?

Unfortunately I have never really used linux, and know almost nothing about it so I can't test my theories. I spoke a bit with Admentus who works on the Zelda64 UHD pack (and other packs), as that pack is also very large in size and takes a lot of RAM to prefetch. Sure enough, he mentioned users have brought this issue to him as well, and a quick search on the discord server reveals this:

Spoiler: (Show Spoiler)

On to the Wind Waker pack. It's probably for the better I ended up working on this "half-size" version as there's a few things that need some manual intervention. Hypatia does not use a single scale factor for all textures. My tool currently isn't equipped to deal with a Dolphin pack where you want to just halve the resolution of each texture (Rescale with 0.5 as Rescale Factor), as if "tex1" is found, it treats the texture different and uses the original resolutions. I had to make a few modifications to create this pack, so I'll have to think about adding some option to enable this next CTT-PS version.

Spoiler: (Show Spoiler)

Some foliage textures with transparency look awful with mipmaps. But you can't just generate the pack with mipmaps disabled, as this game also contains several arbitrary mipmaps, or different mipmaps that show at different camera distances.

Spoiler: (Show Spoiler)

When giving these textures mipmaps, they don't look so good at higher resolutions.



Removing the mipmaps fixes the issue. Don't forget, do NOT use anisotropic filtering with this game!

Hypatia gave me the go-ahead, in fact there will now most likely be a small update soon to all packs which fixes the edges of some textures I missed last time. I also want to update the widescreen patcher I included, the UI is blurry at higher DPI, and I found an alternate widescreen patch (but I'm not sure who created it, so I don't know who to credit..). But I may not use it anyway, only if it fixes the misplaced fire/ice arrow icon. Edit: This ended up being the same patch anyway.

Edit: I managed to get them all uploaded, half-size pack and others can be found on the main post (use the mirror links for now until Hypatia gets a chance to update her site):
https://forums.dolphin-emu.org/Thread-hypatia-s-tloz-the-wind-waker-hd-pack-v2-0001a?pid=483553#pid483553

[sudo]

Wow! Thanks for making that ! Impressive work, and I'm sure other people who simply don't have the ram for it will truly appreciate yours and hypatia's efforts! Thanks a ton! I'll use this and prefetch textures again to see if the problem still happens. Will edit this post with further information as I aquire it.

Any chance the widescreen patcher can work natively on linux and with .rvz Dolphin's compressed rom version? All my roms are this format. I'd love if this tool was capable of working on more games too, I have no idea the intricacies that go into something like that however.

Edit:
So far no issues at all, and the size is crazy small! I dont notice any real difference. Also I bet if you really wanted to, you could combine parts of the two packs (like highest detailed hud) for the best of both worlds, but honestly, I don't think it even matters! Whole pack loads around 2gbs for me. I never expected it to be this small and still look so good!

[Bighead]

The widescreen patch itself was created by gamemasterplc, I just created a GUI around it with PS. Again, I know nothing about linux so I can't help out much there. It's an xdelta patch so if there's a linux version of that it should work fine. If you still have a windows VM up and running, the patcher should work on it without any problems. The only other alternative is to copy your game ISO to a windows machine and apply the patch there. The original link to the patch can be found here:
https://forums.dolphin-emu.org/Thread-unofficial-widescreen-development-topic-archive?pid=452406#pid452406

I've never used Dolphin's RVZ, but IIRC it's lossless so should be able to be reversed. As long as the disc has an MD5 of d8e4d45af2032a081a0f446384e9261b. If I'm wrong about that, unfortunately you may need to redump it.

As for your edit, I'll leave that up to the users if they really want to copy it over but I honestly doubt it's necessary. The current resolution of the UI should be good at least up to 1080p, and even in 1440p it looks really good (my screenshots above are using the half-size pack at 1440p). At that point we're talking about making this pack a "workaround". And while it kind of started that way, it should exist to be more of an all-rounder pack for those with weaker systems/lower resolution screens, which doesn't really justify putting in any of the full size textures.

The real end goal from all this should be figuring out exactly why prefetch crashes in linux, and hopefully it can be fixed in Dolphin. I think I have about enough info at this point to make an issue report (if one doesn't already exist), but it would be nice to figure out exactly what triggers it beforehand so the devs know what to look for.

[sudo]

I simply assumed that there would be issues with the vm version.
I dont have the .iso anymore but frankly it's not a big deal. Stretch is fine and I have no idea how to reverse it. Redump would be a must I'm sure.
I agree it's pretty pointless, I wrote that as just a thought for the people who have this issue but run a 4k monitor or something but want the best possible graphics. Idk man. I play on a television so it's not much if any real difference
Anyways I feel I've bothered you enough with my dumb questions ahah. If there's anything I can do to help provide answers for the bug report lmk.
@sudokek#2564 is my discord

[Bighead]

It got me curious so I looked into it real quick. You should be able to convert RVZ back to ISO with Dolphin. Just right click the game in the list and click "convert". After the patch is applied, it can be converted back to RVS with "convert".

Spoiler: (Show Spoiler)

[sudo]

Oh right on, I guess I was just ignorant of that. Thanks!

[Bighead]

It seems that someone beat me to it by about 12 months, and an issue was filed here:
https://bugs.dolphin-emu.org/issues/11994

I suppose all we can do is wait and hope someone takes it on some day. At the very least, I'll make a post there saying this is still an issue.

[Bighead]

New beta version:
https://www.mediafire.com/file/vbk5p4qwa4ow4xw/Custom_Texture_Tool_PS_v47.0.b29.7z/file

Development has gone a bit underground with some private testing since v47.0.b22, hence the sudden increase to b29. Here is what has changed since then:

• Phyre Mode now has partial support for The Legend of Heroes: Trails of Cold Steel III and IV. The "assets.pka" file can be extracted, as well as the PKG files in these games. The phyre textures can be converted to PNG and back to phyre, support for BC7 has been added, and extracted files can be repacked into PKG files. There is still many limitations to what can be done, first and foremost the games experience the same UI scaling bugs from the first two games, and I doubt we will get a Durante fix this time. The "assets.pka" file can not be repacked, and the PKG files can only be recompressed with LZ77 (CS3/CS4 use LZ4 compression, but still work with LZ77). For more info, I have updated and added a TON of information to this post:
https://forums.dolphin-emu.org/Thread-custom-texture-tool-ps-v46-2?pid=482262#pid482262

• The Process Selected menu accessed with the [$] button next to Start had much of it rewritten since it had many design flaws. There were several cases where texture paths could become lost so textures would fail to be created, especially if using the "Overwrite Textures Directly" option. Generating previews is now nearly as fast as not generating them at all.

• When using the Combine/Split operations, a debug option has been added to increase the maximum values of the Grid Array beyond my limits, and the maximum values have been increased to 120. This limit previously imposed was just a random number I chose to make sure users did not set values too high which can either lock up the GUI for long periods of time or outright crash it. This was requested by the Zelda64 texture pack modders, as there are some textures that resolutions beyond my default limit.

• Since updating Hypatia's Wind Waker packs, I realized I needed a way to downscale Dolphin textures by a % value and CTT-PS wasn't up to the task. Rescale Textures works a bit different if it's a Dolphin texture ("tex1" in filename) than how it works when the filename does not have that prefix. Dimensions of Dolphin textures are calculated using (original dimensions * scaling factor), the original dimensions are pulled from the file name. Dimensions of all other textures use (current dimensions * scaling factor). So if you wanted to downscale the textures in a Dolphin pack by 50% (rescale factor of 0.50), this was not possible and I'm not sure why I never added this until now.

Spoiler: (Show Spoiler)

• Instances of GetFiles in the script were replaced with EnumerateFiles when counting textures. This should provide a speed-up when counting many textures. I have some ideas where this may or may not improve performance in other places, as its 100s of times faster than Get-ChildItem but requires a bit more effort to implement.

• The programs pack was updated with more recent versions. I created a new installer using NSIS script which allowed me to provide a lot more information during installation, and also also allows choosing exactly which programs the user wishes to install. 

[Bighead]

New beta version. I have finally added everything I wanted to for this version (it has taken me over a year!), so new features are on hold until v47.0 is released.
https://www.mediafire.com/file/aqkmw2hnnvecdz8/Custom_Texture_Tool_PS_v47.0.b30.zip/file

(11-12-2020, 03:05 PM)ImDirtyDan Wrote: Just wanted to share something I made. It's a tool that will automatically generate material maps from textures and has a mode that will save them in Ishiiruka's texture tool format. https://github.com/JoeyBallentine/Material-Map-Generator/

The big addition this time is the Material Map Generator, which can be accessed with the operation Create Materials With Material Map Generator which can only be found in Dolphin Mode. This has been on my to do list for several months and it's finally implemented.



Because it requires Python and a number of other Python dependencies, I have added a "Check Dependencies" button that will run through everything to check if anything is missing. If Python or Material Map Generator are missing or they are not linked to the script, it will alert you to go download them. If there is any missing Python dependencies, the script will automatically download them.

There are a few options at your disposal:

Tile Size: I think this is similar to the workaround I implemented in ESRGAN/SFTGAN. If the GPU runs out of memory, smaller tile sizes can work around it.
Edge Pixels: Also similar to my upscaling methods, this can set different ways to handle the edge pixels. I have not tested it enough to understand how "Mirror" and "Replicate" work.
Disable GPU Acceleration: Pretty self explanatory. This shouldn't be used unless the GPU is made by AMD as CUDA is nvidia only.

I have not tested it too much as I do not use Ishiiruka and have never really worked with material maps, but from my limited testing it does seem to work. Before the release I will do much more testing to make sure everything is good to go.



Other changes include updating the ESRGAN/SFTGAN automatic installer so it downloads the latest version of Python and other dependencies. Also fixed some scaling/alignment issues with some GUI elements at 96 DPI or when high DPI scaling is disabled in the script.

[Andre18]

Still working on an Tenkaichi 3 Texture Pack. I still ai upscaled over 18000 Textures AS now. 
But with some Textures I had some Problems.
Post it in the attachment. Did your tool had any Option to fix this automatically? Its hard to find on which of all these 18000 Textures still has that Problems too.