Custom Texture Tool PS v50.1

[Ericthe1]

(05-22-2016, 07:49 AM)Bighead Wrote: Here you go: http://www.mediafire.com/download/d74i31gjsuqdbff/xBRZ+ScalerTest+Batch+v0.1.zip

- Copy script to the folder with your textures.
- Edit with a text editor and set the path for ScalerTest, or copy the ScalerTest folder into Program Files (x86).
- Run it and select the strength you want to apply from 1-6, default is 4 if nothing is entered.

I copied a lot from my old batch version of CTT so it was fairly easy to make. It will also search all sub-folders and keep the hierarchy when creating files in a folder named ~Filtered_#xBRZ.

Awesome, the batch file worked perfectly Bighead. Thanks.

[CyberGlitch]

Thanks for all your work on this Bighead!

I'd be really cool if Waifu2x was included as a scaling option. It can get even better results than xBRZ.

In the meantime I think I can scale the textures manually with a waifu2x batch I have, but I'm worried about artifacts for repeating textures since the scaler doesn't have any pixels on the edges to compare to in order to line up the edges seamlessly. The simple solution would be creating a 3x3 grid of the texture, scaling it up, then cropping out the middle square. Can you think of an easy way to automatically convert to a 3x3 version of the texture like that, and to crop them all after they've been scaled by the Waifu2x batch?

[rlaugh0095]

So if im understanding this right, I can use this to convert from png to dds? I've beeN converting from dumped dds to png, because my editor of choice doesn't support dds and my pack is getting way to big. XD

[Bighead]

(06-07-2016, 06:32 AM)CyberGlitch Wrote: Thanks for all your work on this Bighead!

I'd be really cool if Waifu2x was included as a scaling option. It can get even better results than xBRZ.

In the meantime I think I can scale the textures manually with a waifu2x batch I have, but I'm worried about artifacts for repeating textures since the scaler doesn't have any pixels on the edges to compare to in order to line up the edges seamlessly. The simple solution would be creating a 3x3 grid of the texture, scaling it up, then cropping out the middle square. Can you think of an easy way to automatically convert to a 3x3 version of the texture like that, and to crop them all after they've been scaled by the Waifu2x batch?

I'll look into this "Waifu2x", never heard of it before. And I'm not sure what you mean by 3x3 grid, if you mean literally 3x3 then 9 pixels isn't much to work with. Maybe you mean as in 3x3 pixels per cell in a grid? Example below... a 9x9 texture, broken into 9 cells of 9 pixels each. And you would want to crop out the green section?
    
This would also mean the image would have to have a multiple of 3 pixels on both dimensions. Any specific reason why you chose 3? Am I even on the right track? I'm a little confused on exactly what you want (pics may help) .

(06-07-2016, 02:38 PM)rlaugh0095 Wrote: So if im understanding this right, I can use this to convert from png to dds? I've beeN converting from dumped dds to png, because my editor of choice doesn't support dds and my pack is getting way to big. XD

Yes it can convert between PNG/DDS/JPG (although JPG is pretty useless as textures), and afaik it should now be able to handle every possible case. I wanted to eventually make a list of all cases that I know about and post it somewhere so here it is.

- Convert/rescale any PNG, DDS, JPG file
- Detect corrupted files and copy them
- Internal mipmaps (DDS) or external mipmaps (Dolphins mip1, mip2, etc..)
- Any small textures that got botched with OptiPNG
- Uncombined materials are combined in output path (needs color texture, bump/spec/nrm automatically get converted to combined nrm)
- Combined material maps (needs color texture present or they will go undetected)
- Color texture created with Ishiiruka Tool with missing material (every program/editor thinks the file is corrupt, I use some trickery with Ishiiruka Tool and ImageMagick to automatically restore it to a format that works)
- Ishiiruka-textures-to-Dolphin Internal Option (Ishiiruka Tool DDS color textures seem to fail in normal Dolphin (example is 4k ZTP pack), so an option exists to reconvert them to a format that does work)

For converting to DDS, it needs DDS Utilities. If material maps are present, it would also need the path to Ishiiruka Tool in the Internal Options. I would suggest going in there and setting some preferences such as "DDS MipMap Type" and such.

[CyberGlitch]

2xWaifu does scaling magic by using AI to figure out what the image is supposed to be. It's tuned to work best on cartoon style images, but it also works well on photos. Here's a comparison to other scalers when applied to a 256x256 grass texture: http://imgur.com/Csmzhxs

Here's a gallery of the issue I'm having and how to fix it: https://imgur.com/a/ikFBa

As it says, that grass texture stacks in a 3x3 grid seamlessly, demonstrating how well it works as a repeating texture. Unfortunately, when scaled with 2xWaifu seams are created because the scaler lacks the context of the adjacent edges. The solution is to scale a 3x3 stack of the texture so the scaler can match up the edges, then crop out the middle square to get an HD textures that blends seamlessly with itself. The difference this makes is shown in the last GIF.

[General_Han_Solo]

Bighead, what's going on here?

[Bighead]

(06-08-2016, 06:00 AM)CyberGlitch Wrote: 2xWaifu does scaling magic by using AI to figure out what the image is supposed to be. It's tuned to work best on cartoon style images, but it also works well on photos. Here's a comparison to other scalers when applied to a 256x256 grass texture: http://imgur.com/Csmzhxs

Here's a gallery of the issue I'm having and how to fix it: https://imgur.com/a/ikFBa

As it says, that grass texture stacks in a 3x3 grid seamlessly, demonstrating how well it works as a repeating texture. Unfortunately, when scaled with 2xWaifu seams are created because the scaler lacks the context of the adjacent edges. The solution is to scale a 3x3 stack of the texture so the scaler can match up the edges, then crop out the middle square to get an HD textures that blends seamlessly with itself. The difference this makes is shown in the last GIF.

Oh okay now I see what you mean. It should be possible to do this with ImageMagick (I think). I'll look into it and see if I can come up with something.

Edit: Yes it is possible with ImageMagick "montage" to create a 3x3 grid of the image, and cut out the center with the "-crop" parameter. I came up with a concept that works, now I just need to figure out 2xwaifu and automate it. BTW it would be helpful if you could share that batch script you have just to get an idea of what others are using for 2xwaifu's arguments.

Edit Again: This batch should fulfill your desires, copy the script to the folder with images. It will also work on sub-folders. It must be edited with notepad, the path variable "Waifu2x" set to where the program is, and "WaifuApp" should match the application (exe). ImageMagick should be pulled from the registry. It is assumed you are using the command line version of waifu2x (I found many different versions), the one I used is linked below. I did not use any of the options that comes with it, so consider this a "test version", but the scale is able to be changed. I'll add the ability to tweak the other options a bit later.

Script: http://www.mediafire.com/download/3x9o61i6949bk7k/Waifu2x+Seamless+Texture+Batch+v0.1.zip
waifu2x: https://github.com/tanakamura/waifu2x-converter-cpp

Edit More: This one exposes a few more options, not sure if its important to include setting the processor, threads, and block size (whatever that is...).
http://www.mediafire.com/download/3p92e1zb003cr6e/Waifu2x+Seamless+Texture+Batch+v0.6.zip

(06-08-2016, 06:12 AM)General_Han_Solo Wrote: Bighead, what's going on here?

It appears PowerShell v2.0 Rename-Item doesn't support the "-LiteralPath" parameter. Not sure which version it was added. Try this and see if you still get the error:

I'll have to officially fix this in the next version. Using Rename-Item is a pretty recent thing, it's probably better to use Move-Item since it can work the same...

Edit: Here is the better "fix" for older PowerShell versions. The issue in 8.1 left garbage mipmap textures laying around that failed to be renamed, so you might want to delete the pack you were generating and start over.
http://www.mediafire.com/download/9d20m2d8dxbksfm/Custom+Texture+Tool+PS+v8.2.zip

Edit Again: I also wanted to say the reason this error came to existence was because I added an alternative to how DDS mipmaps are created. It is faster with both ImageMagick and DDS Utilities to create the mipmaps internally. Before I would always create all mipmap levels as individual images for external mipmaps, and/or combine them together with nvidia "stitch.exe" for internal mipmaps. Now I always create the mipmaps internally and extract them with "detach.exe" if external mipmaps are selected. This shaves off a few seconds when converting mipmap textures which adds up into the minutes for big packs. The only time I use the old method now is if "Force New Mipmaps" is disabled (useful only if the pack has dynamic mipmaps - different images on descending layers, which so far is very rare). This also has the advantage of creating DDS files with internal mipmaps that work with GIMP-DDS.

[General_Han_Solo]

It works now!

[CyberGlitch]

(06-08-2016, 06:26 AM)Bighead Wrote: Edit Again: This batch should fulfill your desires, copy the script to the folder with images. It will also work on sub-folders. It must be edited with notepad, the path variable "Waifu2x" set to where the program is, and "WaifuApp" should match the application (exe). ImageMagick should be pulled from the registry. It is assumed you are using the command line version of waifu2x (I found many different versions), the one I used is linked below. I did not use any of the options that comes with it, so consider this a "test version", but the scale is able to be changed. I'll add the ability to tweak the other options a bit later.

Edit More: This one exposes a few more options, not sure if its important to include setting the processor, threads, and block size (whatever that is...).
http://www.mediafire.com/download/3p92e1zb003cr6e/Waifu2x+Seamless+Texture+Batch+v0.6.zip

Thank you so much for the quick solution!

There are a number of versions of Waifu2x. I've had the best performance with waifu2x-caffe, which was built to allow GPU acceleration with Nvidia's CUDA cores (it greatly increases the speed to scale images). The version you chose as added this feature as well, which is very useful since conversation otherwise can take ages. Caffe currently seems to be bit faster if you have a modern Nvidia GPU (750 Ti or newer), but CPP runs better on other graphics cards that don't support CUDA acceleration.

Most of the settings for Waifu2x can be ignored. The most important one to adjust is the "Noise reduction level," which cleans up noise and compression artifacts in an image but reduces the level of detail. For most textures I'd suggest disabling noise reduction by setting it to 0. Most people always want to double the image size (set "Scale" to 2). To quadruple the size you have to run the scaler a second time. Generally you don't need to mess with the block settings. Waifu2x breaks the image into blocks, converts them piece by piece, and puts them all together again. Tweaking these settings might slightly improve performance for your system, but it's only really necessary if the block size ends up so large that a section consumes all of your RAM (causing the program to crash).

Since only the seam of the edges matter, this could be greatly optimized by cropping the 3x3 grid before scaling to leave just a little "overlap." Here's a few images to visualize this: http://imgur.com/a/R1jTp
The white rectangle marks the original texture in the grid. The red rectangle marks how the 3x3 grid should be cropped (5/12ths the 3x3 grid's size to leave 1/8th of the outside squares). Scale this cropped image with 2xwaifu, then crop again to get the final seamless texture (by cropping by 4/5's the upscaled image). This would make it work ~6x faster.

[Bighead]

(06-09-2016, 04:27 AM)CyberGlitch Wrote: The most important one to adjust is the "Noise reduction level," which cleans up noise and compression artifacts in an image but reduces the level of detail. For most textures I'd suggest disabling noise reduction by setting it to 0.

In the version of waifu2x that I am using (cpp) I noticed the noise reduction value isn't applied unless the mode is set to "Noise" or "Noise-Scale", so setting it to "Scale" is the only way of disabling it since it won't allow a value of 0 or it gives an error. Similar situation for "Scale", it will not be applied unless the mode is set to "Scale" or "Noise-Scale". I think the reason it was done that way so that "Mode" can act as a sort of master toggle for the values and everything can be done on one line like this:
"%Waifu2x%\%WaifuApp%" %NoGPU% %OpenCL% --scale_ratio %Scale% --noise_level %Noise% -m %Mode% -i "!MontyPath!" -o "!WaifuPath!"

(06-09-2016, 04:27 AM)CyberGlitch Wrote: Most people always want to double the image size (set "Scale" to 2). To quadruple the size you have to run the scaler a second time.

The scaler setting allows setting values higher than 2, even odd numbers like 5, are you saying processing the image multiple times in increments of 2 gives better results? Obviously this wold only work for 2x, 4x, 8x, 16x, and if it does give better results its probably worth implementing it for these specific integers.

(06-09-2016, 04:27 AM)CyberGlitch Wrote: Since only the seam of the edges matter, this could be greatly optimized by cropping the 3x3 grid before scaling to leave just a little "overlap."

Brilliant idea, that would indeed greatly decrease the amount of time it takes to process the image.