[UNOFFICIAL] Ishiiruka-Dolphin Custom Version

[StripTheSoul]

Will the PNG files get a similar treatment or is there no more room for speedup for them?

[Tino]

for png is really difficult to do something, png artist create exelent textures, but they do not know about programing and memory management, if you create a big amount of rgb textures and you load them directly without talking in acount memory consumition in a specific frame, you end with a bloated frame that consumes huge amuount of resources and destroy the performance. is really dificult to fix that from the programer size, the easy fix will be resize all the textures to smaller sizes to get the performance back.

[neobrain]

Uh really, VRAM consumption is the least problem that our custom texture code has. The issues are well-known among the developers, there's just no one who has had time and interest to fix them, yet. DDS just masks away the problems in the loading algorithm and the texture management code.

EDIT: Obviously point still stands that those dds builds happen to have their followers, yet I'd like to stress that it's not super-hard to fix the code properly.

[Tino]

neo: can you point what are from your perspectives the problem that hires textures presents?

[StripTheSoul]

@ Tino: BTW, just wanted to clarify that the missing body parts textures issue in Xenoblade Chronicles ( https://forums.dolphin-emu.org/Thread-unofficial-testing-needed-dx9-final?pid=319927#pid319927 ) has nothing to do with costum textures (also happens when the feature is not activated). I just had a cutscene with three people missing their faces. That's creepy, man ^^

[Tino]

don't worry i already see the problem will try to fix it soon when I merge a more stable core

[StripTheSoul]

Not worrying, just reporting ^^
Thanks, though!

[Bighead]

Hey all, I'm basically maintaining and updating the Xenoblade texture pack for the time being (with tons of help from other users), and StripTheSoul brought a few things to my attention from this thread. So, I figured it would be a good idea to come here and see if I can clear up a few small concerns with custom textures. I'm sorry if I'm hijacking the thread and throwing it off topic, atm this seems to be most relevant place to ask these questions, so I'll try to keep it short and contained. If anyone would care to answer them, it will be greatly appreciated!

Concerns..: (Show Spoiler)

(05-01-2014, 12:20 PM)Tino Wrote: the problem you have with xenoblade was caused by some textures with incompatible size. now they are skipped. To fix those textures they must be resized to valid multiple of 4 sizes.

This is something I actually fixed a few weeks ago for the PNGs (and apparently, am still randomly finding new ones to fix lol), but the current PNG pack does not have the rescaled versions yet. Now what I'm wondering, do the dimensions of custom textures have to be an exact multiple of the original texture? I'm not very good at explaining this... so for example, base texture is 32x32, would a retexture scaled to something like 128x132 cause any non-obvious issues I don't know about (other than squishing it vertically)? I ask because it's very possible that those before me did not keep the original scale in mind for some of these textures.

(05-03-2014, 05:50 AM)Tino Wrote: for png is really difficult to do something, png artist create exelent textures, but they do not know about programing and memory management, if you create a big amount of rgb textures and you load them directly without talking in acount memory consumition in a specific frame, you end with a bloated frame that consumes huge amuount of resources and destroy the performance. is really dificult to fix that from the programer size, the easy fix will be resize all the textures to smaller sizes to get the performance back.

I won't argue that point. I've looked at Dolphin's code and it's definitely a work of art in its own form. A lot of amazing work by extremely talented people. StripTheSoul has mentioned to me he has major performance issues with PNG textures (and in the current pack, half of them don't even load because of the weird dimensions). While I don't know much about programming, I can say with certainty going through and scaling down 1000's of textures is a lot more work than it would seem, especially when trying to keep the integrity of their quality. Most textures in the Xeno-pack are between the 128x128 - 256x256 range, with some in the 512x512 range, and very few exceeding that. So I'm wondering, is a lot of the problem with PNG performance really the scale? Many of the textures that the game has been dumping for environments have larger scales than most of the retextures in this pack.

(05-03-2014, 06:15 AM)neobrain Wrote: Uh really, VRAM consumption is the least problem that our custom texture code has. The issues are well-known among the developers, there's just no one who has had time and interest to fix them, yet. DDS just masks away the problems in the loading algorithm and the texture management code.

^ This kind of already answers my last question (but not completely), so my question here is an extension of my last question. If the issue with PNG performance is Dolphin related and not so much the scale of the textures themselves, and it was fixed (no pressure here, I understand this is low priority and the devs have bigger things to worry about), would PNG performance match or at least come close to the current performance of DDS files? This is just something I want to keep in mind for the future. When the next iteration of the pack is complete, I want to avoid releasing a DDS version of it if possible, but that's probably unlikely with the current state of things. I want everyone to be able to experience it to the fullest.

I think that's pretty much everything that I've been wondering about. And so sorry for the wall of text. I tried to keep it short, but I failed. Also, thanks for fixing texture loading in D3D neobrain!

[Tino]

Bighead:
no problem with te post i'm really happy to answer.
Texture scale:
the best scenarios is that the hires textures maintain a scale in respect to the origial textures, for example a original texture of 32x32 replaced with a 128x128 one is perfect, but they will stiil work if they don't, if the size if not a multiple of the original the texture is still loaded and displayed with no issues.
the only basic rule for compatibility with all the backends is that width and height are multiples of 4, for example in you current version there is a floor texture that has 1001px with, for png is a valid size (but in some hardware implementation is slower) but for dds on d3d9 backend is an invalid size.
Texture Size:
the problem is not that the textures are to large, you have to hidden problems with that.
first format change: game developers cleverly chosse texture formats to reduceresource usage and improve performance. most textures are stored used compresed formats, IntensityAlpha or palleted textures.
when you replace them with png all is encoded in rgba so for example for a 32x32 IA texture you use 32x32x2(bytes per pixel) = 2KB is you replace that with a 128x128x4(rgba bytes per pixel) = 64Kb that is and increment of 3200% in size, but is not a problem newwer hardware can handle that without problem, BUT if you start to think not in a single texture but in a complete frame you start to see the problem, if you put in a single frame a complete set of new textures you can blow away the amount of memory needed. and event on good hardware you will see performance drop faster when that happends. So my point is, is not bad to make large textures, but you have to be smart and don't exagerate because and exess in size will kill performance. the golden rule of game developers is, don't use resources for thinks that the user will not see, if you have an ui icon that will display in a hd screen you know the covered area, so use that resolution, don't make it to big because it will be a waste of resources.
about Neobrain issues:
in my implementation don't worry about the custom texture loading code I already rewrite it and still optimizing it, will try to maximize performance to take png path alittle closer to dds path.
about the dds pack: the generation of dds packs is really easy with tools from amd and nvidia like "The Compressonator" you can transform entire forlders in minutes and event generate mipmaps automatically so is not a problem to generate and entire set from the png files

[Bighead]

Thank you very much for taking the time to answer my questions, it was very informative. I definitely don't want to convert all PNG textures to use indexed channels because there is just too much quality loss there. There are some that could come down in scale though, I'll have to take a good look through them. In the end, to me it seems useful to still offer a DDS version of the pack. And it is easy to generate DDS, that was not my concern. I've been using Aorta which can also generate mips, and I'll definitely check out "The Compressonator" (lol at the name). My concern was that I was under the impression DDS may be dropped someday from Dolphin and I didn't want people to download a pack that will eventually not work for them. If that day ever does come, they could just get online and DL the other one anyway.

Again thanks that was very helpful and cleared up a lot!