Custom Texture Tool PS v50.1

[e2zippo]

Amazing Bighead! That was fast!

[Bighead]

Rather than elaborate on my previous posts, I figured I'd start over since this has been a complex task and I have no idea where I left off. I shoved both of those posts into a single one above.

The options are ready to be played with, the download is here:
http://www.mediafire.com/file/bnz1fndt0a5jm9g/Custom_Texture_Tool_PS_v43.0.b1.zip
http://www.mediafire.com/file/uzfwhsl9k8xgkw3/Custom_Texture_Tool_PS_v43.0.b2.zip

Combine Multiple Textures: (Show Spoiler)

This section explains the changes to the Combine Multiple Textures operation.



Grid Array: The number of columns * rows in the grid.
Inner Pixel Overlap: The number of pixel columns/rows that overlap between textures.
Button Ratio: The shape of the button. Buttons are 100x100 pixels. A ratio of "1.00 : 0.50" means 100x50 pixels.
Output Name: The name of the combined texture when created.
Select Textures: Opens the grid to assemble textures.
Load CTT File: Load a previously combined texture. Auto-fills the grid with textures (if they are found).

---

The texture grid has also seen some updates.



Done: Finish adding textures and close the window.
Clear: Clear all textures from the grid, now with verification window.
Arrows: Moves a selected texture around the grid.
Hide Names: Hides all names on the grid. Button becomes "Show Names" when pressed.
Font Invert: Toggles the texture names black/white.

---

Textures can now be selected in the grid and moved around using the arrow buttons, arrow keys, or WASD. A selected texture will have a green highlight around it.



To deselect a texture, just click it again. There is currently a bug with selecting them that causes selected buttons to sometimes be slightly "bigger". This is due to buttons and tiles not tiling/overlapping 100% perfectly. This has no impact on operation, but I'll hopefully fix it before release.

---

Some examples of the texture grid when altering "Grid Array" and "Button Ratio".

Grid Example Previews: (Show Spoiler)

Grid: 4x4, 1.00 : 1.00 / Grid: 4x8, 1.00 : 0.50

Split Combined Multi-Texture: (Show Spoiler)

This section explains the changes to the Split Combined Multi-Texture operation.


Load CTT File: Load a previously combined texture.
Texture Path: Load any texture to be split.
Grid Array: The number of columns * rows the texture will be split into.
Insert Overlap Pixels: The number of extra overlap pixels that will be inserted between each texture.

When combining a texture, the overlap pixels should be taken note of. This is always "1" for Zelda OoT box textures. When splitting a texture, the insert overlap pixels should be multiplied by the number used when combining. So if the new image is 8x, then overlap pixels in the case of Zelda which is always the upscale value, should be (1 * 8 = 8). If overlap was 2x for another game, they should be (2 * 8 = 16).

CTT File (created when combining): (Show Spoiler)

A CTT file (.ctt) is created whenever textures are combined. This is a text document that holds extended information about the textures.



It holds valuable information that can be used to reload all the segments into the texture grid, or split the image and automatically rename all segments back into their original names. If the path and texture names are known, a CTT file "could" be manually forged.



Line 1: The name set in "Output Name" when combining.
Line 2: The original path to the texture segments. They must all be located here.
Line 3: Dimensions of the original segments. They must all match these dimensions.
Line 4: The number of columns used.
Line 5: The number of rows used.
Line 6: The number of overlapping pixels between segments.
Line 7: The button size that was used when combining.
Line 8+: The list of textures. They are loaded along rows before columns.

Pretty straight forward stuff, its just a means to store stuff to be used later. When loading a CTT file, a few things should happen:

• The number of rows and columns will be imported into both operations.
• The fully combined texture should always be loaded into the "Split" operation.
• If the path to the segments has not changed since they were combined, they will be reloaded into the "Combine" operation grid.
• The pixel overlap in the "Combine" operation is loaded directly, the pixel overlap in "Split" operation is calculated from original dimensions.
• A CTT file is immediately released when loading with "Combine", it is held onto to prevent options changing when loading with "Split".

There are some special nuances to the CTT file:

• If the Line 1 output name does not match the texture current name, it will instead try to use the name of the CTT file.
• Line 2 segments path can be edited to use a path relative to the CTT file by starting with "..\". For example: if the CTT file and texture is in "C:\Combined", and the path on Line 2 is set to "..\Segments", the script will look for segments in "C:\Combined\Segments". This allows creating a single folder backup of the segments and combined texture, and a single CTT file will load them all.

Zelda: OoT Stuff: (Show Spoiler)

By now the idea has been pretty well established. When combining, overlap the duplicate pixels. When splitting, replicate the duplicate pixels. Everything works the way I intended it to work.

So far I have been successful and have done all I set out to do. But I noticed one final imperfection...



The textures above are waifu2x vs. original. Where the walls of the boxes meet, there is a visible seam. This is even present in the original textures, so it is not an issue of getting the alignment correct.

While it "could" be possible to fix this, it would take much more effort than an automated tool could fix. This goes into the territory of fixing "developer bugs". The exact boundaries of where textures meet would have to be found, the images themselves would have to be slightly "squished" to accommodate extra pixels, and pixels from the previous image would have to be copied over into the freed up space.

It's not an awful amount of work, but it would be annoying and may never be perfect. Here is how this "could" be fixed.

Step 1: After combining segments to create the 4 images, combining the 4 images into 1 image, upscaling that image, and splitting back into 4 images, we would be left with our "final" 4 upscaled images to be re-split into segments.



Step 2: Each image would need to be squished to be slightly "thinner". This depends on how many pixels are being cut off between the seams of the boxes.



Step 3: A bit of each texture would need to be duplicated between edges, an amount equal to what is needed to make it seamless in-game which would require some trial and error.



Step 4: Split the textures back into segments which would normally be done.

It "may" be possible to automate this, but I will have to experiment more and it may even have to be its own option or script. The difficulty comes in the fact the original textures must be manipulated to be "thinner". This is fixing developer bugs, so it could also have its own issues. If the overlap between the walls are big enough, it could throw off the image relative to the geometry.

Edit: After much messing around, the closest I could come is this:



Upscale was 4x size. I squished the upscaled images by 6 pixels width: 1012 > 1006, then copied 3x lines from each next texture into the previous. I tried 2x and 4x and both were farther off, so this is as good as it can get.

That should cover everything, the main features are done. From here on out will mostly be bug fixes and maybe a few more QoL features if I can think of any.

@Admentus: I figured I'd also share all my test files and conditions which will hopefully help in some way.
http://www.mediafire.com/file/uw2ke5d9pioqd89/Zelda64_Upscale_Stuff_(upload.01).7z

[Admentus]

Nice work Bighead!

I should make preparations some time to study how it works and get to up-scaling those backgrounds myself .

Through I suspect that will be taking a few days, if not weeks before I am set to go. For now I am focused on improving Ocarina of Time and Majora's Mask through the magic of AR Coding. And I suspect Nerrel will soon be finished with Ikana Canyon. So that takes priority too.

But I have made some preparations to get started with this incredibly dull task of up-scaling backgrounds. At least your tool prevents me from having some serious migraines.

[Bighead]

If you want I'll upscale and split the hyrule market backgrounds as I already have them plotted out. All of the work is basically done for those except the actual upscaling. Just let me know what upscale settings you want (filter, model, factor, etc).

Also it's all not as complex as it may seem, I just wanted to cover every possible option and use case. In the case of these hyrule market "skybox" like textures, if you were to do it from scratch, the entire process would be...

- Open "Combine Multiple Textures".
- Set grid array to 4x8, set button ratio to 1.00:0.50, set inner pixel overlap to "1".
- Open the grid and fill it in. This is the tedious part of figuring out the puzzle. Plotting a texture map helps.
- Press "Start" to combine them, it creates a "combined texture" and a CTT file.
- Upscale the "combined texture" that popped out. Keep the upscaled texture and CTT in same path.
- Load the CTT File in "Split Combined Multi-Texture", the upscaled texture is loaded and all necessary settings are applied.
- Press "Start" to split the texture. They will automatically be renamed and ready to be loaded in Dolphin.

There are the other tweaks that could be performed, and if you wish for me to do the market textures I'll try to eliminate the box seams as much as I can.

[Admentus]

(08-13-2019, 09:14 PM)Bighead Wrote: If you want I'll upscale and split the hyrule market backgrounds as I already have them plotted out. All of the work is basically done for those except the actual upscaling. Just let me know what upscale settings you want (filter, model, factor, etc).

Also it's all not as complex as it may seem, I just wanted to cover every possible option and use case. In the case of these hyrule market "skybox" like textures, if you were to do it from scratch, the entire process would be...

- Open "Combine Multiple Textures".
- Set grid array to 4x8, set button ratio to 1.00:0.50, set inner pixel overlap to "1".
- Open the grid and fill it in. This is the tedious part of figuring out the puzzle. Plotting a texture map helps.
- Press "Start" to combine them, it creates a "combined texture" and a CTT file.
- Upscale the "combined texture" that popped out. Keep the upscaled texture and CTT in same path.
- Load the CTT File in "Split Combined Multi-Texture", the upscaled texture is loaded and all necessary settings are applied.
- Press "Start" to split the texture. They will automatically be renamed and ready to be loaded in Dolphin.

There are the other tweaks that could be performed, and if you wish for me to do the market textures I'll try to eliminate the box seams as much as I can.

Ok. I will be trying my hand at it. I am sort of finished with AR coding in order to improve Ocarina of Time and Majora's Mask for the time being. I have been a bit busy today, so I am sorry for the late reply.

My intentions are an 8x upscale with ESRGAN, and see if that works. The previous backgrounds seemed to have turn out quite well with ESGRAN. Basically it should be running the ESGRAN upscale progress twice. If it all works smooth and fast, I can be with it in no time. There are a lot of backgrounds to cover. The easier and more convenient this is the faster I can get everything in place of course.

I love to try out the process myself, so I would get skilled enough to do it myself. That doesn't mean no one is allowed to help out of course .

So far I am loving the Combine Multiple Textures GUI. And your posted tutorial is perfectly clear. It couldn't be any easier to instruct. If I manage to get any results out of it, I'll post a screenshot of it.

And thank you for your hard work! Really! You're a life saver.

So I did my first test, using ESGRAN twice. Very easy with your tool. The only thing that would take time it making the jigsaw. But for testing purposes I only skipped that phase. I managed to successfully going to the subsequent tasks. Really, it couldn't be any easier. You praticially only need someone willing puzzling these jigsaws together, and even that is easier than ever with your Custom Texture Tool. If you set aside the jigsaw puzzling, you can basically get this done within a few minutes.

Hyrule Castle Town Market - Child Day (Front): (Show Spoiler)

I should be trying some of the other filters or scalers. If you have any suggestions, please. Through I like to keep it at a minimum of 8x.

If the process is that easy, then I wouldn't need to set week aside for it. Just the thought of dread working on these backgrounds just made me want to put it all away, and just kept working on the next best thing that has reasonable results. I still won't be able to finish everything within days, as I still have other matters to attend to and I don't expect to get started right away with it yet, but it's definitely a motivating start for me.

Of course I will make sure to include your name in the credits for the contributions towards these backgrounds for the next Zelda 64 4K Texture Pack update.

In the meanwhile I have been experimenting with the 60 FPS interpolation patch from Kaze Emanuar by packing it into a WAD. It definitely works on the Virtual Console, and the beautiful thing is that it keeps all RAM addresses intact for all those AR Codes. I guess Kaze made the patch a while before, but I didn't pay any real attention to it yet. It definitely makes Super Mario 64 feel smoother. I wasn't able to made an AR code out of it. So... That won't happen. But at least it is patchable and functional on the Virtual Console. So yeah, that took my attention among other matters away.

By the way, I am not sure if you did notice it yet, but 3DS emulator Citra is making progress with custom textures support, allowing for texture packs as well there.

[Bighead]

Ooh that looks much better than waifu2x, which I was only using because its fast for testing purposes. Aye, with this implementation complete the worst part is now figuring out the pieces rather than figuring out the overlap. A giant leap in convenience, but still tedious. Like I mentioned before, the easiest way to figure it out is to dump the textures, then run them through the watermark option and load them in Dolphin.

Spoiler: (Show Spoiler)

Also, don't ever feel rushed or obligated by me to work on anything if I happen to work on a feature that lines up with your project (especially since at this point its still experimental). When I read back my own posts, sometimes I feel I may give off this impression, and I apologize if I do. My only goal is to expand the capabilities of this tool, and any time I can see an area where I can make something tedious even slightly easier through some interface, I can't resist the temptation. Especially if it's a feature that hasn't been done before, or isn't offered in any other software. The reward of challenging myself and being able to pull it off is where almost all of my satisfaction comes from (BC7 + arbitrary mipmaps, I'm looking at you especially), so I don't ever want anyone to feel like my efforts go to waste if I work on a feature that nobody makes use of. I'll admit, my reasons for improving it (and even starting it in the first place) are rather selfish, working on it is a guilty pleasure, and the fact it's actually used by others who find it useful is a really a huge bonus.

So now that you have some experience with these new options, I am open to any ideas of improvement. There are a few changes that I still want to do:

- Better lower button arrangement on grid (done, clear, arrows, etc). The arrows should be moved down to the bottom row, and maybe a "deselect" texture button since I like having multiple ways to do the same tasks (ultimately makes things more intuitive).
- Fix the "too large" selection bug, more specifically, fix the button sizes so selection bug doesn't happen.
- Loading a CTT file locks it in place for "Split" textures (a product of the previous design), and releasing it loses the list of names. I'd rather just keep it always loaded until a new one is loaded or the script is closed.
- Redesign the Split Textures operation menu. The red text message isn't necessary, and I'd like to put a preview on there so its known which texture is currently loaded. The "Load CTT File" and texture path textbox/button [...] could probably be merged into a single 'thing'.
- Many of the new buttons and other things need proper tooltips.
- Different button sizes based on ratio. Better config options maybe, automated maybe, not sure. using like 1.00 : 0.10 is kind of weird since they are so small.
- Rather than "Invert Colors" only allowing black and white on the grid, actually pick whatever color you want.
- I'm sure I'll think of more stuff....

I don't want to make any big promises, so don't get hopes too high, but I do have an idea that could "auto-build" these specific textures. We know they are unique in that they have a duplicate row of pixels where they overlap. So it may be possible to scan the textures, scan the edge pixels, and figure out where they go based on that. This could "auto-build" them without knowing the texture map ahead of time. It's an ambitious idea that has been rattling around in my head, and I'll probably see if it's possible after I get everything else done.

---

Edit: This update takes care of many of the issues mentioned above.
http://www.mediafire.com/file/dgk2uavgd3vm28q/Custom_Texture_Tool_PS_v43.0.b3.zip/file

- Texture grid selection is now correct (button sizes were fixed).
- Combine textures dialog has 2 new buttons: deselect and replace (same as left/right click when texture is selected.
- Combine textures dialog buttons were modified to fit better.
- Combine textures dialog font color can now be specified beyond black/white.
- CTT File is no longer held/released when loading with "Split" operation, it just always remains loaded.

Still a few more things to do, there will either be 1 more or no more more "betas" before it's a wrap.

Spoiler: (Show Spoiler)

[Admentus]

Just to demonstrate how quickly and effective your tool is.

Hyrule Castle Town Market - Child Night (Front): (Show Spoiler)

I made it in about 5 minutes. Texture dumping, combining, upscaling and splitting included.

Included the non-scaled version below.

[Admentus]

So this got me thinking about those prerenderedbackgrounds of Ocarina of Time and the Custom Texture Tool.

Ocarina of Time has only two types of backgrounds.
- Type 1: 32 textures with the size of 64x32, into four different sets for a total of 128 textures. This type has been successfully covered in the Custom Texture Tool.
- Type 2: 40 textures with the size of 324x8, this works a bit differently which I will be explaining.



Type 2

Type 2 consists out of 40 textures, being 1 column and 40 rows, making for a combined image of 320x240. On Project64 these 40 textures are dumped in the format of 320x6 but Dolphin dumps those in the format 324x8. Type 2 is also need in other Nintendo 64 games such as Super Smash Bros. The VC version on Dolphin also dumps these backgrounds in the format of 324x8.

Somehow the GameCube versions has 4 additional pixels in the width on the right and 2 additional pixels on the bottom. Fun thing is, these additional pixels are repeats from the last actual pixels (right and bottom) from the format 320x6. Thus these additional pixels are more or less garbage. Still Dolphin does need the textures to be formatted in 324x8. The solution is quite simple, but tedious.



Process for Type 2

- Reduce the canvas size of all 40 textures from 324x8 to 320x6, removing pixels on the right and bottom.
- Combine all 40 textures together to form 1 column and 40 rows.
- Upscale the combined texture.
- Split the combined upscaled texture into 40 textures, being 1 column and 40 rows.
- Extended the canvas size for all 40 textures from (320 x upscale factor) x (6 x upscale factor) to (324 x upscale factor) x (8 x upscale factor), adding pixels on the right and bottom.
- Example: an upscale of 6x would be setting the canvas size from 1920x36 to 1944x48.
- Create a duplicate layer of all 40 textures with a mirror to horizontally to be placed on the empty extended canvas size. Do the same with mirroring it vertically. So the new transparent pixels are being filled with each texture mirror mirrored upon it.
- Done.

The Zelda 64 4K texture pack already has a few upscaled type 2 backgrounds found in the Backgrounds folder.

Could the Custom Texture Tool also account for this process?

[Bighead]

I don't see why not, the details and the process itself doesn't seem too complicating. The hardest part would be figuring out the presentation since this would be highly specialized towards this one situation, and does not line up with any of the currently existing options. It could also mean more changes to the interface and CTT files as it may require more (or different) information. I'll have to mess around with them myself to get a feel for how it should work as I tend to have some strange methods for it to all make sense in my mind.

It is odd to me that the textures are different sizes between N64 and GC versions, I'd be interested in the technical details as to why that is as it seems like an important question to ask. I doubt they are that way in the ROM data, so where are these extra duplicate pixels coming from? Is it a behavior of the Gamecube/Wii hardware? Is it a bug/behavior in Dolphin itself? Is the N64 emulator somehow missing these pixels and they are actually there? Is it the emulator that Nintendo is using for these titles? And if these pixels were put there intentionally by a human mind somewhere along the way, why?

Also, I wonder if these extra pixels are why these textures look so awful at higher resolutions, since it would affect their filtering/tiling. It raises all of the questions above.

Spoiler: (Show Spoiler)

It may be awhile before I make progress, so I may as well release this now since I've completed everything regarding the previous task.
http://www.mediafire.com/file/35j973hqai7tv52/Custom_Texture_Tool_PS_v43.0.b4.zip

- Textures can now be completely removed from the grid by holding control and clicking on them or control + the "deselect" button.
- Selection color can be changed by holding control when clicking "Font Color".
- Tooltips have been added to all dialog items which also explains their hotkeys and control+hotkey combinations.
- Fixed button names appearing when selecting a texture and names have been toggled off.

Spoiler: (Show Spoiler)

[Admentus]

I do have three suggestions

- Allow the "Combine Textures" interface to be resized. Which makes it easier to look at on my 4K screen if it is larger.
- Being able to drag one texture into another, swapping both their places.
- Being able to load in a X amount of textures at once. For example, if the grid is 4x8 (total of 32), allow to import 32 textures at once into the grid. I don't care about the position on the grid just yet. I sort it out on the grid. I don't want to manually drag every texture by one by one into the grid.

and two bugs to report

- The Tool Path: "Tools Master Path (Optional)" is not carried over when using the "Import Stored Options" on a previous version which has been set up.
- The Tool Path: "Temporary Folder" is not carried over when using the "Import Stored Options" on a previous version which has been set up.