multiview rendering 45 views?

[blas_t_dozer]

I'd like to put Dolphin into one of these holographic devices:



It would mean I would have to extend Dolphin's stereo mode to more than 2 eyes--for these devices potentially up to 45 views, in a horizontal rail.

I was looking through the code at places where StereoMode is used, like:

Code:

static u32 CalculateEFBLayers()
{
 return (g_ActiveConfig.stereo_mode != StereoMode::Off) ? 2 : 1;
}

It seems, given the way the camera can already be moved with the stereo hack, that what I'm after would be possible! My questions for the devs:

1. Would performance be reasonable? With the way the geometry shader duplexes out the stereo views, can we guess about how rendering 45 views of a single GameCube frame on a beefy graphics card would do?
2. Is it possible to modify the projection matrix of the camera before rendering?
   

Any thoughts at all are welcome before I go down the road of building Dolphin, etc. Thanks!

[iwubcode]

I hadn't heard of this but I always loved stereoscopic 3d. This is really cool. Do you own one of these? I'd love to hear general impressions if you do. It seems like 3ds type technology but with more pop?

I was very excited but saw the cost. Much like the hololens, it's likely not consumer ready yet. Still neat tech.

(05-18-2020, 02:20 PM)blas_t_dozer Wrote: Would performance be reasonable? With the way the geometry shader duplexes out the stereo views, can we guess about how rendering 45 views of a single GameCube frame on a beefy graphics card would do?

This would be my main concern. I can't speak to this exactly but can say that 3d takes a hit obviously when rendering 2 views, I can't imagine 45!

You can read about the theory here: https://dolphin-emu.org/blog/2015/05/13/a-second-perspective/ . I believe the geometry shader is actually used to do the actual rendering of the two views at the moment. Unfortunately, I haven't really looked into how that works in detail, so I can't can't give you any more specific information.

(05-18-2020, 02:20 PM)blas_t_dozer Wrote: Is it possible to modify the projection matrix of the camera before rendering?

Yes, that's what the freelook camera does but that might be more work for your use case than using the geometry shader.

--

If you do manage to get something working. Please post about it!

[blas_t_dozer]

Yeah! It's 45 views instead of 2 so it definitely has way more of a feel of realism than a DS screen. Not to mention they are bigger and brighter.

I'll keep you posted!

[iwubcode]

Slightly different use case but you can see the changes I made for something similar (rendering 2 views in a split-screen style): https://github.com/dolphin-emu/dolphin/pull/8832

[blas_t_dozer]

An update: Dolphin builds on my Windows machine fine, but then always crashes with a mysterious stack--it must be the JIT compiler, because when I enable the cached interpreter, things run, but way too slowly to be usable. Unfortunately I don't have time to track down JIT crashes right now, so I'll have to come back to this project when Dolphin master is building and running successfully!

[JosJuice]

(05-27-2020, 02:24 AM)blas_t_dozer Wrote: An update: Dolphin builds on my Windows machine fine, but then always crashes with a mysterious stack--it must be the JIT compiler, because when I enable the cached interpreter, things run, but way too slowly to be usable. Unfortunately I don't have time to track down JIT crashes right now, so I'll have to come back to this project when Dolphin master is building and running successfully!

It's not actually a crash. Just tell Visual Studio to ignore it and it'll work.

[blas_t_dozer]

(05-27-2020, 02:48 AM)JosJuice Wrote: It's not actually a crash. Just tell Visual Studio to ignore it and it'll work.

 doh. thanks!

[blas_t_dozer]

You can follow my fork on github - today I refactored the framebuffer manager so it understands that there may be N views, instead of 1 or 2, and got most of the way there in the shader generator as well.

I also added some fancy new holographic-display-related options to the "Enhancements" tab in Graphics Settings:



Unfortuantely I still haven't figured out why, but I'm still only getting 2 views max, even when I try to render 16 in a row, like here: 



There may be more assumptions in the geometry shader that I'm missing...I'll keep this thread posted!

[blas_t_dozer]

An update: I did get Dolphin to render N views instead of 1 or 2 successfully.

Now all that's left is to build in support for a special shader that combines the multiple views into a special format used by the display.

(Am I right in thinking the rules of this forum prohibit me from posting images of running games?)

[JosJuice]

(05-29-2020, 03:35 AM)blas_t_dozer Wrote: (Am I right in thinking the rules of this forum prohibit me from posting images of running games?)

No, that's fine.