Custom Texture Tool PS v50.1

[StripTheSoul]

(05-23-2015, 09:19 AM)Bighead Wrote: Could you also upload that mario sunshine texture? The pack that is currently offered is in the old format and I only need the problematic texture.

Yep, here's the problematic texture: http://www.file-upload.net/download-10635951/bad--bad-texture.rar.html

[Bighead]

After some reading and tinkering I realized the problem lies within a texture size limitation with DDS Utilities. The documentation says that it can support textures up to 4000x8000 resolutions, but it actually seems much smaller. A resolution of 3000x6000 works but not much higher than that crashes. It affects both the size of the input image when read, and the specified output size when attempting to write.

Fixing it for resizing textures + converting to DDS should be easy which would benefit most users, the problem here lies in the input image only. A temporary image with the new dimensions could be generated with ImageMagick without issue (I think) if a really high resolution is detected, rather than just feed DDS utilities the huge one. This would add more time to the process for large textures, but in theory should work and prevent the crash when using the resizing option. I currently have it limited to 16x, the largest dimension I've seen so far on an axis is 512, which results in 8192 and will crash. It may be wise to drop this limit down to something like 8x since anything higher is impractical to begin with. At the very least, I hope to get resizing working without crashes.

Fixing it for 1:1 conversion of huge textures is most likely not possible, and this is where I'm stumped on what the best course of action is. The Mario Sunshine texture linked is 32x the original texture 128x256 >> 4096x8192. Same story with some of those Star Wars textures, while they are only 8x the base is 512x256 which results in the same dimensions. While I find this to be over the top, it's not my textures and I don't want to just force certain sizes when converting, at least not without some kind of option to do so. Another route is to alert the texture will not work rather than just blindly process it, move or copy it somewhere for manual conversion, or just log it, maybe even in a separate "failed" log. I'm open to suggestions on this one.

There's also the possibility of exploring other command line based DDS tools, but not many exist. ImageMagick is great for PNG and reading image data, but support for writing DDS files is limited and buggy. The tools by nvidia are awesome other than the crash because it can extract+combine mipmaps from multiple images and not just force them to be generated. I just stumbled onto this image tool earlier today, but so far from my limited reading it doesn't look like it can do everything I need. I could probably just combine yet another tool into the mix for this edge case, but 3 requirements is already far too many.

If I can't figure out anything, then meh. DDS writing was just a perk of this script to begin with, as my original intentions were to just write a script to find/log texture errors and got carried away.

Edit: Leaving all that rant up there in tact, I realized the issue was trying to rescale textures with DDS Utilities. Generating a temporary texture with the correct dimensions in ImageMagick, and then feeding that into DDS Utilities is what fixed the problem.

@General_Han_Solo: This should not be a problem depending on the scaling factor, and whether or not you have DDS auto correction enabled. This option calculates the dimensions of the new texture based on the original dimensions * the custom texture width integer value ignoring any decimals. For example..

Original: 640x384 Custom: 2112x1267 Scale Factor: 3.3
In this case, both the width and height are scaled the same but only the width scale is checked, and is then multiplied to the original texture width and height. The width integer scale here is 3, the decimal is dropped, and the result is: 1920x1152 This ensures the texture has a proper aspect ratio and integer scale. In the odd case something does not land on a multiple of 4, it is also scaled up to the nearest. This very slightly throws off the aspect/scale by 1-3 pixels, but nothing can be done about that.

Disable DDS auto correction, and it just scales dimensions up to the nearest multiple of 4. The result will instead be 2112x1268, which is how other batch DDS programs do it. This gets you closer to the original dimensions, which is useful for small scales like 1.87 (which would become 1 with correction), 2.5 (which would become 2), etc... but does not yield an integer scale, and it does not respect the aspect ratio. The Tales of Symphonia pack for example needs this option disabled or the textures end up looking like shit.

[Bighead]

All DDS crashes hopefully fixed and generated DDS mipmaps are now much higher quality. Also fixed a few bugs I missed.
http://www.mediafire.com/download/m4rew88os4z78w2/Custom+Texture+Tool+v3.2.zip

StripTheSoul will you test this again on packs that crashed? More specifically "Option 5".
Han Solo will you try this on your Rebel Strike pack now and see if you still have issues? More specifically "Option 6".

[masterotaku]

I converted my texture pack (Zelda Collector's Edition) and there's something I didn't expect to happen: there are some DDS files that are a lot bigger than their PNG version! I just chose the option to convert to DDS, with the slowest compression mode.

Big files with incorrect aspect ratios (I increased the canvas horizontally for 16:9 aspect ratio) ended up upscaled in one axis instead of downscaled to the previous multiple. The most funny example is a 5111x3571 PNG (5684KB) that was converted to a 4964x4624 DDS file (22416KB). Most of the image is just transparent space, so I don't understand how a lossy file can take so much space.

Did I do something wrong? Am I crazy for using such high resolution textures?

Other than that, original textures that I just corrected for 16:9 (small textures from the menu) look like crap, but I can just provide their PNG counterpart.

[StripTheSoul]

@ masterotaku: It's normal that the DDS files are bigger... I think it's because of the included mipmaps. That doesn't mean, though, that they're loading as slow as PNGs.

@BigHead: Cool, I'll try it out, thanks

[StripTheSoul]

Hm, first off, I tried that one big problematic texture from the SMS pack. It did go through without crashing that nvidia exe, however, I then got the textures with all those mipmaps lying around seperately, so it seems they weren't included in the DDS file. They should be, though, right?

This is the log:
tex1_128x256_m_66f988b1dcbd9332_14.dds [Old] 128x256 [New] 512x1024 [Scale] 32.00x32.00 [Forced] 4.00x4.00 [Status] Converted

[Bighead]

That sounds right. Currently both internal and external mipmaps are generated for DDS because for all I know master branch still doesn't use internal ones. This is more or less a compatibility feature. A bug prevented them from sometimes being generated in the previous version of the script. External mips were only generated if it found at least 1 PNG mipmap (mip1, mip2, etc...) and that was definitely not my intention. If the textures are to be used in Ishiiruka only, you can delete the external mipmaps if you really want but they shouldn't harm anything. If master branch ever utilizes internal ones, I'll update the script to no longer generate them.

Sounds good though, I just recently tested converting both Star Wars packs without resizing and no crashes. So I think it's fixed.

Edit: That also reminds me I have to update the log for DDS because that information can be offputting. "Old" reflects the original texture dimensions, "new" reflects the DDS texture generated, and the "scale" is from the custom texture.

[StripTheSoul]

Aaaah, ok, I thought the mips were only external now. That's ok then

[Bighead]

(05-29-2015, 02:38 AM)masterotaku Wrote: Big files with incorrect aspect ratios (I increased the canvas horizontally for 16:9 aspect ratio) ended up upscaled in one axis instead of downscaled to the previous multiple. The most funny example is a 5111x3571 PNG (5684KB) that was converted to a 4964x4624 DDS file (22416KB).

This is because of DDS auto correction, it uses the original texture dimensions * integer scale instead of your PNG texture dimensions. If you do not want this to happen, then disable it in the script. [color=red]set DDSAutoCorrection=false[/color] Your texture should instead end up as 5112x3572. There is no clean way to scale "down", because there is no way to know when to do so. No program will do this (at least not Aorta or the Compressonator), they, along with this script without correction, scale up 1-3 pixels so dimensions end up as a multiple of 4. Those programs also can't create mipmaps at 1x1 and 2x2, but this script can.

Edit: I should also add, that because your textures are not created with the original dimensions in mind, using the option to convert textures to a fixed integer scale will not work correctly with this pack because the result is calculated from the original texture dimensions. Disabling DDS auto-correction and using option 6 is as good as it gets in this case for a near 1:1 conversion. I guess I could technically make an option to create textures at some fraction of the size of the custom texture, but that would just encourage bad textures which is the opposite of what this script is supposed to do.

(05-29-2015, 02:38 AM)masterotaku Wrote: Most of the image is just transparent space, so I don't understand how a lossy file can take so much space.

Some textures have mipmaps included in the texture (m flag in the name) as well as the external ones. DDS is larger because they are essentially the size that they will occupy in vram. They do however compress in rar, zip, and 7z much better than PNG. This forum post has some relevant information on DDS sizes: http://stackoverflow.com/questions/19651510/how-to-calculate-how-much-video-memory-or-memory-cost-by-a-texture

IIRC, the formula we care about it is width*heigh*pixel size, pixel size depends on the channels (RGB=3, RGBA=4). Somebody correct me if I'm wrong, a 1024x768 RGBA texture would be 3145728 bytes, or exactly 3MB. Textures are also compressed with DXT1C (no alpha channel) or DXT5 (alpha), which I'm not sure if it will affect the size of the DDS or the size in vram. It appears that for DDS file size you can just do width*height, because the texture you mentioned earlier (4964*4624) ends up as 22415.5625 KB. I don't fully know the correct way to calculate the sizes, degasus posted a formula somewhere awhile back but I can't remember now.

[masterotaku]

Thanks for the explanation, Bighead. I may try those settings tomorrow.