Is it feasible to pre cache the shaders?

[Thotaz]

As it is now Dolphin caches shaders as you are playing through the game, the problem with this is that it causes small freezes in the games (at least on my system). This obviously makes the experience of the first playthrough quite bad, so I'm just wondering if it would be possible to make it cache these shaders in advance? Also are the caches unique to your system? If they aren't then it might be worth it to backup caches from a complete playthrough, maybe even share them with other users.

[Sonicadvance1]

If we had some sort of system in place to gather anonymous usage statistics from games we could have a system that would generate a database of used shader IDs and have people download the shader IDs per game and let the user decide if they want to have it pre-compile all the shaders for that particular game on first run.
That's just a idea that I happened to think about.

Another one could be that instead of having shader caches separate per game, have a combined shader cache for all games so the chance of hitting a shader that isn't yet compiled is lessened.
This idea has been rejected though.

[Qaazavaca Qaanic]

Am I correct in saying that an unofficial implementation was recently added to the DX9/Ishiiruka branch?

[Thotaz]

(07-28-2014, 07:20 AM)Sonicadvance1 Wrote: If we had some sort of system in place to gather anonymous usage statistics from games we could have a system that would generate a database of used shader IDs and have people download the shader IDs per game and let the user decide if they want to have it pre-compile all the shaders for that particular game on first run.
That's just a idea that I happened to think about.

Another one could be that instead of having shader caches separate per game, have a combined shader cache for all games so the chance of hitting a shader that isn't yet compiled is lessened.
This idea has been rejected though.

It would probably have to be opt in then, because the cache can grow quite big from what I can tell; the one I have for metroid prime 2 in openGL takes up 10.9MB and I only played around for a few minutes near the start of the game. If you are on a crappy internet connection then uploading 500MB or whatever a fully cached game uses might be a little too much to just happen without you being aware of it.

[JMC47]

Jimbo1qaz: No, there isn't any implementation of that in his build. That's Vertex Loader stuff; which is mainly because the Vertex Loader JIT sucks.

[Sonicadvance1]

(07-28-2014, 08:15 AM)Thotaz Wrote:

(07-28-2014, 07:20 AM)Sonicadvance1 Wrote: If we had some sort of system in place to gather anonymous usage statistics from games we could have a system that would generate a database of used shader IDs and have people download the shader IDs per game and let the user decide if they want to have it pre-compile all the shaders for that particular game on first run.
That's just a idea that I happened to think about.

Another one could be that instead of having shader caches separate per game, have a combined shader cache for all games so the chance of hitting a shader that isn't yet compiled is lessened.
This idea has been rejected though.

It would probably have to be opt in then, because the cache can grow quite big from what I can tell; the one I have for metroid prime 2 in openGL takes up 10.9MB and I only played around for a few minutes near the start of the game. If you are on a crappy internet connection then uploading 500MB or whatever a fully cached game uses might be a little too much to just happen without you being aware of it.

You misunderstand my random idea. It would save the shader IDs which are 64bit apiece, not the full shader binary. We can't upload shader binaries since they are different between vendors and even families of GPUs.
Every shader ID a game would use would most likely be less than 512KB

[rexoe]

what if we had an option to prerender cache before the game starts????? :^)

[Shonumi]

(07-30-2014, 12:58 PM)rexoe Wrote: what if we had an option to prerender cache before the game starts????? :^)

You need to run the game in order to generate the shader. There's no way to make the shader before you play the game because the shader is based on what you play.

[BigT]

(07-28-2014, 07:20 AM)Sonicadvance1 Wrote: If we had some sort of system in place to gather anonymous usage statistics from games we could have a system that would generate a database of used shader IDs and have people download the shader IDs per game and let the user decide if they want to have it pre-compile all the shaders for that particular game on first run.
That's just a idea that I happened to think about.

This sounds like a pretty good idea for a community project, which sounds feasible to me... that should at least take care of the more popular games.

[JMC47]

It's not really feasible, but it is something. There is a huge challenge to it, bandwidth, multiple videocards/drivers would constantly invalidate it. New builds would often break old shader caches as well.