asynchronous shaders questions

[DolphinPC]

(07-19-2015, 11:17 PM)MaJoR Wrote: Async shaders and the uber shader are pretty much the only techniques known... The shader compilation is the problem, and there is no way to control it since they are a part of the D3D and OGL spec. It's either continue the game without waiting for the shader compilation (async shaders) and get visual errors, or do an uber shader and work around the problem.

Btw, by "better performance" I meant "no more stuttering". The uber shader technique would most likely be more demanding on your PC hardware. But it would fix an unfixable problem without any added issues (in theory) so it might be worth it.

Well if that's the only possible fix without any errors or issues (gameplay/visual/sound), it really worth trying, and I really hope that someone will try to implent it and try, because besides that, at least the Gamecube versions of MP1 and 2 have almost perfect emulation.

[Fiora]

(07-19-2015, 01:54 PM)Aleron Ives Wrote:

(07-19-2015, 01:02 PM)Fiora Wrote: The Ishiruuka developers simply refuse to contribute the feature, and nobody else has volunteered to port it from Ishiruuka.

From what I've read, this isn't the case. I thought one or more of the developers for master were against the asynchronous shaders, because they fix the stuttering during shader cache generation by making various game assets simply become invisible if they haven't been generated yet, and said developer(s) thought the speed boost afforded by such an approach was too hackish and buggy.

I've never seen anyone on the dev team say that? Certainly it'd be optional as a feature, but I don't know anyone who's against it. I don't know where this myth keeps coming from. Maybe it's because people want to blame the dolphin devs instead of the people who refuse to contribute their work?

Personally I prefer the ubershader -> swap shaders after async compilation approach, but you could get there by starting with async shaders and working forward.

[DolphinPC]

(07-20-2015, 12:05 AM)Fiora Wrote:

(07-19-2015, 01:54 PM)Aleron Ives Wrote:

(07-19-2015, 01:02 PM)Fiora Wrote: The Ishiruuka developers simply refuse to contribute the feature, and nobody else has volunteered to port it from Ishiruuka.

From what I've read, this isn't the case. I thought one or more of the developers for master were against the asynchronous shaders, because they fix the stuttering during shader cache generation by making various game assets simply become invisible if they haven't been generated yet, and said developer(s) thought the speed boost afforded by such an approach was too hackish and buggy.

I've never seen anyone else on the dev team say that? Certainly it'd be optional as a feature, but I don't know anyone who's against it. I don't know where this myth keeps coming from. Maybe it's because people want to blame the dolphin devs instead of the people who refuse to contribute their work?

Personally I prefer the ubershader -> swap shaders after async compilation approach, but you could get there by starting with async shaders and working forward.

Then if someone can contact the people who created async shaders, maybe he should. it can get dolphin closer to perfection in some games, and like you said, who knows how it can help if the creator would help to implent it on an official release, and let the developers try to develop it further.

[Aleron Ives]

(07-20-2015, 12:05 AM)Fiora Wrote: I've never seen anyone on the dev team say that? Certainly it'd be optional as a feature, but I don't know anyone who's against it. I don't know where this myth keeps coming from.

My source was this post by delroth, which I will paste here for convenience:

(02-28-2015, 07:03 PM)delroth Wrote:

(02-25-2015, 06:25 AM)AnyOldName3 Wrote: Master-branch Dolphin is an accuracy-focused emulator above all else, and if some devs had their way, everything would be done with interpreters and software renderers to ensure perfect accuracy in every respect except speed. There are definitely settings in Dolphin that make things faster or prettier at the risk of breaking other things, but there aren't a huge number of them - it's a nuisance to have to look after two code paths for everything, as it potentially means that every change has to be made twice in two different places in two different ways. This slows down development.

Oversimplifying the issue doesn't help anyone. No, it's not "Ishiiruka likes speed" and "Dolphin devs like slow things". In fact I hope you realize that's extremely offensive to people who have worked for years on improving Dolphin's performance. The real issue is that async shader compilation is a horrible hack that partially breaks rendering for an undefined amount of time at undefined time in the execution. As it is it probably is the most broken hack in the history of Dolphin. It's PCSX2 level of hacky.

There are more elegant solutions to this problem (delaying shader compilation and using a slower macroshader as an alternative until shaders are compiled, for example). Nobody seems interested enough in the problem to implement that solution.

At least one Dolphin developer thinks asynchronous shaders are too hackish for master and that any speed improvements in shader generation should be achieved through more polished means. Is the "macroshader" concept the same thing as the "uber shader" discussed in this thread?

[NaturalViolence]

Fiora Wrote:I've never seen anyone on the dev team say that? Certainly it'd be optional as a feature, but I don't know anyone who's against it. I don't know where this myth keeps coming from. Maybe it's because people want to blame the dolphin devs instead of the people who refuse to contribute their work?

I have seen several dolphin devs say it before.  I just hopped on IRC quickly to confirm:

Quote:[15:39]<NaturalViolence> is anyone here legitimately against asynchronous shader compilation being implemented?
[15:39] <@HdkR> I still am, even in -dev
[15:40] <NaturalViolence> quiet you
[15:40] <Bh44L>
[15:44] <+JMC47> NaturalViolence, it'd be a nightmare with issue reports and whatnot
[15:44] <+JMC47> and would pretty much kill any motivation to fix it right
[15:44] <+JMC47> so, I'm against it as well
[15:44] <+JMC47> (note, for generic/uber shaders, asynchronous generation of faster shaders would still be used)
[15:44] <@flacs> what's the difference?
[15:45] <+JMC47> flacs, when a shader isn't done, Dolphin doesn't draw the graphics but marches on
[15:45] <+JMC47> that's what asynchronous shader generation does
[15:45] <+JMC47> except with some things, they're only drawn for one frame so Mii faces and shit don't work at all
[15:46] <@HdkR> JMC47: async compilation of specialized shaders*
[15:46] <NaturalViolence> it would be an option
[15:46] == dogen [~dogen3@50-0-50-11.dsl.dynamic.fusionbroadband.com] has joined #dolphin-dev
[15:46] <NaturalViolence> also there are different ways to implement it
[15:46] <NaturalViolence> as seen in the unofficial builds
[15:46] <@HdkR> NaturalViolence: The uber shader approach is the route we want to take
[15:47] <NaturalViolence> so then is anyone opposed to uber shaders?
[15:47] <@flacs> no
[15:47] <@HdkR> (It's not a hack)

And there were a couple more that agreed later on that async shader compilation is too hackish but uber shaders would be ok.  That seems to be the general consensus.

[DolphinPC]

(07-20-2015, 05:29 AM)Aleron Ives Wrote:

(07-20-2015, 12:05 AM)Fiora Wrote: I've never seen anyone on the dev team say that? Certainly it'd be optional as a feature, but I don't know anyone who's against it. I don't know where this myth keeps coming from.

My source was this post by delroth, which I will paste here for convenience:

(02-28-2015, 07:03 PM)delroth Wrote:

(02-25-2015, 06:25 AM)AnyOldName3 Wrote: Master-branch Dolphin is an accuracy-focused emulator above all else, and if some devs had their way, everything would be done with interpreters and software renderers to ensure perfect accuracy in every respect except speed. There are definitely settings in Dolphin that make things faster or prettier at the risk of breaking other things, but there aren't a huge number of them - it's a nuisance to have to look after two code paths for everything, as it potentially means that every change has to be made twice in two different places in two different ways. This slows down development.

Oversimplifying the issue doesn't help anyone. No, it's not "Ishiiruka likes speed" and "Dolphin devs like slow things". In fact I hope you realize that's extremely offensive to people who have worked for years on improving Dolphin's performance. The real issue is that async shader compilation is a horrible hack that partially breaks rendering for an undefined amount of time at undefined time in the execution. As it is it probably is the most broken hack in the history of Dolphin. It's PCSX2 level of hacky.

There are more elegant solutions to this problem (delaying shader compilation and using a slower macroshader as an alternative until shaders are compiled, for example). Nobody seems interested enough in the problem to implement that solution.

At least one Dolphin developer thinks asynchronous shaders are too hackish for master and that any speed improvements in shader generation should be achieved through more polished means. Is the "macroshader" concept the same thing as the "uber shader" discussed in this thread?

Actually I have no idea, but it's too bad that no one is interested enough in the problem, besides the shader stuttering, MP1 and 2 are really close to perfect emulation (maybe some minor issues), and developing such a solution would really improve their gameplay experience, making them a lot more fun to play. I really hope that in the future someone will try to implent it, it sounds like it has a great potential.

[AnyOldName3]

It might not be too great to rely just on that one delroth quote (although we don't have to because of NV's IRC log) as he was mostly telling me off for saying something I'd intended as a joke, and he'd thought I meant seriously, so was already going to be overly negative when he started writing.

[DeeKay]

One thing I'm confused about is the need to dynamically compile shaders for each game. If the Wii and GC's GPUs are fixed-function, then shouldn't there be a finite number of shaders needed to emulate all of the functionality? Unless this is what the uber-shader thing is supposed to cover?

[AnyOldName3]

There're lots of components, each of which is fixed function, so while the maximum number of things that can be combined to make something happen is finite (or at least countable), the number is huge, and it would add a ridiculous amount to Dolphin's disk usage if it prepared every possible shader individually. This is also mostly why the ubershader isn't a thing yet - there are a lot of cases it has to cover.

[MayImilae]

Actually the issue is that it's impossible to pre-render all of the shaders. With PC games they create the shaders ahead of time, but with the GC/Wii there are no shaders and the games call on whatever they want from the hardware live as they run. This creates the shader generation stuttering, but also means that Dolphin can't know what the game is going to do until the game does it!

The cache stores the already made shaders so that the second time a game does something, it will be instant. That's the best Dolphin can do without an uber shader.