So I'm trying to write a user manual for Dolphin 3.5, since it'd be a good exercise in technical documentation (plus I like writing, a lot) and it might benefit the community. Of course it's strictly unofficial atm. It's going to offer some technical insight into the inner workings of Dolphin, but not so much as to overwhelm the average reader. A document like this needs to be accurate, however, to be useful. There are parts of the emulator that I'm a bit sketchy on, so if any devs or knowledgeable users have any info about the following questions/topics, I'd appreciate your input. Don't worry about getting too detailed either; be as verbose as you want.
1) - JITIL Recompiler - From what I understand, this stands for Just-In-Time-Intermediate-Language, correct? This takes native GC/Wii code for the PPC-based CPU, recompiles it into another intermediate language (doing some optimizations?) before finally recompiling this into x86 or x64 assembly, correct? What exactly is this intermediate language?
2) - Enable Screen Saver - If you click on the Config button and go to the Wii tab, there's an option to Enable Screen Saver. What I have written so far reads : "When checked, allows the computer's screen saver to activate while running Dolphin." I'm largely curious as to if that's true. It just seems like a strange option to have, especially in the Wii section.
3) - DVD Root & Apploader - If you click on the Config button and go to the Paths tabs, you can specify paths for the DVD Root and Apploader. I just want to clarify what the DVD Root and the Apploader are. If you extract the files from an ISO, you get a directory structure that mirrors what's on the DVD you dumped, so Dolphin lets you specify the "root" directory of the game, effectively allowing you to boot games using these extracted files, correct? I have no idea what the apploader is. Useful for Homebrew on Dolphin or something?
4) - Scaled EFB Copy - On a technical level, what exactly is the Embedded Frame Buffer and why does scaling a copy of it improve graphical quality?
5) - Ignore Format Changes - Does what it says on the tin, i.e. ignores format changes to the EFB. I'm a bit curious as to how often format changes happen (and why); also, for confirmation, this option is always enabled on OpenGL, but it can be disabled on DX9 and DX11, correct?
6) - XFB - What exactly is the External Frame Buffer, and what are the technical differences between Real and Virtual?
7) - Show Drives & Purge Cache - Misc. curiosity, under the View menu in the toolbar, what does the Show Drives option actually do? Purge Cache, I assume deletes the code cache, shader cache, both, or something else?
8) - Texture Cache Accuracy - On a technical level, what is the texture cache, and how do you make it more/less accurate?
There'll be more as I progress with the manual, but thanks to anyone willing to respond.
(01-22-2013, 04:06 AM)Shonumi Wrote: [ -> ]2) - Enable Screen Saver - If you click on the Config button and go to the Wii tab, there's an option to Enable Screen Saver. What I have written so far reads : "When checked, allows the computer's screen saver to activate while running Dolphin." I'm largely curious as to if that's true. It just seems like a strange option to have, especially in the Wii section.
lol, afaik that has to do with the wii screen saver XD, but then again that's just what I always assumed
(01-22-2013, 04:17 AM)ExtremeDude2 Wrote: [ -> ]lol, afaik that has to do with the wii screen saver XD, but then again that's just what I always assumed
The Wii has a screen saver? News to me
EDIT: I don't play my Wii and leave it sitting there for long periods of time, so I've never seen the Wii screen saver in real life. You don't need to have a NAND dump or anything to do this on Dolphin, right? It should work with any Wii game?
(01-22-2013, 04:06 AM)Shonumi Wrote: [ -> ]6) - XFB - What exactly is the External Frame Buffer, and what are the technical differences between Real and Virtual?
I still have this from my notes:
Quote:<neobrain> NaturalVi: XFBs are just some chunks of RAM which are usually sth like 640x576 in size (for both PAL and NTSC and whatever else)
<neobrain> usually games only use one or two of them, they're used for moving stuff to the screen then: Game renders stuff to the EFB, copies it to the XFB, moves it to screen
<neobrain> when copying from EFB to XFB vertical stretching (plus filtering) can be applied. When moving stuff from XFB to screen, horizontal stretching is applied.
<shuffle2> VI technically supports other things too. gamma and horizontal stretching, plus other stuff
<neobrain> shuffle2: yeah, just that we want to keep it simple for NaturalVi for now xD
<shuffle2> lol
<neobrain> fwiw, the EFB uses other pixel formats than XFBs...the EFB uses RGB formats, whereas the XFB is YUV (or whatever it's called)
<neobrain> and XFBs obviously can directly be modified via the CPU additionally.. if one feels like doing that
<neobrain> now to Dolphin's implementation:
<neobrain> "virtual XFB" uses VRAM textures to emulate the XFB stuff as far as possible (doing a limited amount of the fancy stuff shuffle2 already mentioned, too), i.e. rather than displaying the EFB directly to the screen, it's first copied to another texture which then is displayed to the screen
<neobrain> (there can be multiple XFBs/textures involved in that process, but whatever..)
<neobrain> in contrast, when using real XFBs the EFB gets encoded to YUV(whatever) to the RAM (similiarly to "EFB to RAM") and when the VI is requested to display the XFBs' contents to the screen we decode it again to a texture which is then displayed
<n00b421> BTW what games do need xfb's?
<n00b421> never used this option....
<shuffle2> mostly homebrew because they sometimes write directly to xfb
<shuffle2> sometimes other games use it for effects or to display two halves of an AA'd image
full log of that discussion (might just be confusing, idk) :
Quote:<neobrain> erm, kind of. yea
<BhaaL> gnah, olly kills the exe, and ida sucks at debugging
<BhaaL> any alternatives, anyone?
<neobrain> NaturalVi: why? ..
<NaturalVi> neobrain: explain it to me
<neobrain> lol
<NaturalVi> everything
<NaturalVi> difference between virtual and real
<NaturalVi> whether it defines the backbuffer resolution
<neobrain> NaturalVi: k, then the real hw behavior first
<NaturalVi> or scales to the backbuffer resolution
<neobrain> NaturalVi: XFBs are just some chunks of RAM which are usually sth like 640x576 in size (for both PAL and NTSC and whatever else)
<-- Luigi__ has quit (Read error: Operation timed out)
<neobrain> usually games only use one or two of them, they're used for moving stuff to the screen then: Game renders stuff to the EFB, copies it to the XFB, moves it to screen
<NaturalVi> from what I understand
<NaturalVi> efb at 640 x 528 (assuming fullscreen)
<neobrain> when copying from EFB to XFB vertical stretching (plus filtering) can be applied. When moving stuff from XFB to screen, horizontal stretching is applied.
--> n00b421 (~steffen@port-92-196-11-165.dynamic.qsc.de) has joined #dolphin-emu
<NaturalVi> is copied to the xfb
--> skid_au (~fbff@220-253-180-174.NSW.netspace.net.au) has joined #dolphin-emu
<shuffle2> VI technically supports other things too. gamma and horizontal stretching, plus other stuff
<NaturalVi> scaled to 640 x 480, 720 x 480, 720 x 576, depending on the type of game/screen
<-- Hydro has quit (Read error: Connection reset by peer)
<neobrain> shuffle2: yeah, just that we want to keep it simple for NaturalVi for now xD
<shuffle2> lol
* EmuDev gives channel operator status to skid_au
<NaturalVi> I would think frame merging is done as well
<shuffle2> i want accurate AA emulation neobrain! go!
<NaturalVi> since it ouputs 24 fields per second over composite
<NaturalVi> even if it renders 30/60 fps
<neobrain> NaturalVi: when copying from efb to xfb you can only do vertical stretching, horizontal stretching is done after that..
--> aztec (felipefpl@200.97.138.228) has joined #dolphin-emu
<neobrain> whatever frame merging is, it's above my scope I guess... 
<NaturalVi> take all the pixels in a frame
<NaturalVi> put it into an array
<NaturalVi> take all the pixels in another frame
<NaturalVi> put it into another array
<NaturalVi> add the two arrays
<NaturalVi> divide by 2
<NaturalVi> you get an average
<neobrain> ah. ok
<NaturalVi> that becomes the new frame
<neobrain> no idea about that one, but whatever.. that stuff above was the general process 
<NaturalVi> so you can take 60 fps
<NaturalVi> and tun it into 30 by buffering and merging
<NaturalVi> I am GUESSING/ASSUMING that this is done in the xfb
<n00b421> calculate the amount of memory when using 32bit rgba in 1920x1080 and tell me encoder which is capable of doing that...
<neobrain> now let's not go off topic kthx -.-
* Tails removes ban on *!luis*@*.shellium.org
* EmuDev sets ban on *!luis*@*.shellium.org
<NaturalVi> because when I play mp3 on the wii
<NaturalVi> i know the game runs at 60 fps in dolphin
* neobrain will just keep on talking
<NaturalVi> over composite the wii is outputting 24 fields per second
<neobrain> fwiw, the EFB uses other pixel formats than XFBs...the EFB uses RGB formats, whereas the XFB is YUV (or whatever it's called)
<NaturalVi> and when I am moving around is has the unmistakable look of frame merging
<neobrain> and XFBs obviously can directly be modified via the CPU additionally.. if one feels like doing that
--> Hydro (~Hydro@pool-209-158-4-170.altnpa.east.verizon.net) has joined #dolphin-emu
<neobrain> now to Dolphin's implementation:
<neobrain> "virtual XFB" uses VRAM textures to emulate the XFB stuff as far as possible (doing a limited amount of the fancy stuff shuffle2 already mentioned, too), i.e. rather than displaying the EFB directly to the screen, it's first copied to another texture which then is displayed to the screen
<neobrain> (there can be multiple XFBs/textures involved in that process, but whatever..)
<neobrain> in contrast, when using real XFBs the EFB gets encoded to YUV(whatever) to the RAM (similiarly to "EFB to RAM") and when the VI is requested to display the XFBs' contents to the screen we decode it again to a texture which is then displayed
--> Luigi___ (Luigi__@AMontsouris-158-1-16-108.w92-128.abo.wanadoo.fr) has joined #dolphin-emu
<NaturalVi> ok from what I understand
<-- Hydro has quit (Read error: Connection reset by peer)
<NaturalVi> I don't know if dolphin works like this but
<NaturalVi> in every video game I have ever seen a framebuffer called the backbuffer contains the "final image"
<NaturalVi> and the backbuffer contents are sent to the monitor at each refresh
<neobrain> yes, that's what XFBs boil down to be used for as well
<NaturalVi> so does dolphin copy the xfb contents into the backbuffer
<NaturalVi> than send the backbuffer contents to the monitor?
<n00b421> it needs to be copied into the backbuffer, otherwise you would see the copy action
<-- sanchez has quit (Remote host closed the connection)
<n00b421> or the drawing whatsoever
--> Hydro (~Hydro@pool-209-158-4-170.altnpa.east.verizon.net) has joined #dolphin-emu
--> sanchez (~em@66.98.134.21) has joined #dolphin-emu
<NaturalVi> if so does using real xfb set the backbuffer resolution to the xfb resolution
<NaturalVi> or scale it?
<neobrain> erm, scale it
<neobrain> but the resolution will still effectively be the native xfb resolution
--> RME (~RME@136.167.188.28) has joined #dolphin-emu
<AngerFist> hola sanchez
<neobrain> hm.. having a hard time right now finding out where the real XFB's contents get sent to screen in Dolphin...
<NaturalVi> but it does get scaled to the backbuffer resolution?
<neobrain> obviously
<Daco> shuffle2: remember the memory card talk from yesterday?
<neobrain> keeping the aspect ratio though
<n00b421> neobrain: isn't that done in render.cpp Swap?
<shuffle2> Daco: ?
<neobrain> n00b421: meh.. I was about to say sth like "don't you say I would've figured that much if it was that easy?" but then I noticed that you're right.. -.-
<neobrain> if (g_bSkipCurrentFrame || (!XFBWrited && (!g_ActiveConfig.bUseXFB || !g_ActiveConfig.bUseRealXFB)) || !fbWidth || !fbHeight)
<neobrain> seriously, why make it that complicated?
<Daco> shuffle2: about how cool it would be to finally have a memory card tool that could make raw backups of memory cards which then ofc are usable in dolphin (or just to keep backups of an unstable card * looks at his shit card*)
<shuffle2> i think there is a bit of a problem where VI processes frames even when it's not supposed to
<NaturalVi> now THAT is an if statement
<shuffle2> you can see it in some parts of games where there are supposed to be black screens. at boot and stuff
<BhaaL> someones gotta make a statement at some point 
<neobrain> if (g_bSkipCurrentFrame || !(XFBWrited || !(g_ActiveConfig.bUseXFB && g_ActiveConfig.bUseRealXFB)) || !fbWidth || !fbHeight) would be kind of easier
<shuffle2> so dolphin does XFB processing even tho there is no demand for it
<neobrain> the middle part, that is
<neobrain> and more logical, too
<BhaaL> you just changed its meaning, neo
<NaturalVi> all I know is for me both real xfb and virtual xfb have never worked properly for any game in any build
<shuffle2> no, that's someone's law...forget the name
<shuffle2> Daco: um just do it if you want to?
<Daco> i did
<shuffle2> ok
<Daco> and i love it
<shuffle2> yay
<n00b421> BTW what games do need xfb's?
<n00b421> never used this option....
<-- PsyMan has quit ()
<neobrain> <neobrain> and XFBs obviously can directly be modified via the CPU additionally.. if one feels like doing that | hm, seems like we aren't even emulating that with real XFB ...
<shuffle2> mostly homebrew because they sometimes write directly to xfb
<neobrain> BhaaL: yes, it should actually be if (g_bSkipCurrentFrame || !(XFBWrited || (g_ActiveConfig.bUseXFB && g_ActiveConfig.bUseRealXFB)) || !fbWidth || !fbHeight) ...
<shuffle2> sometimes other games use it for effects or to display two halves of an AA'd image
<neobrain> XFBWrited is such a naming fail lol
<shuffle2> hm, seems like we aren't even
<shuffle2> emulating that with real XFB ...
<shuffle2> ^nope, that works
<shuffle2> don't think it works with virtual xfb tho
--> PsyMan (~none@athedsl-244694.home.otenet.gr) has joined #dolphin-emu
<sl1nk3> It doesn't, virtual XFB doesn't copy anything in ram
<sl1nk3> And that's a shame because codec communications in MGS twin snakes are broken just because of a dump XFB effect
<shuffle2> just enjoy the beauty of real xfb :>
<sl1nk3> ahah
<neobrain> meh, still wondering where stuff gets decoded to VRAM again...
<neobrain> but yeah, I was also under the impression that it should work, right now is just the first time I'm looking at how it actually is implemented
<n00b421> neobrain: Maybe this time I'm wrong.. but the show should start in BPMEM_TRIGGER_EFB_COPY in BPStructs.cpp
<n00b421> even vs2k10 should be able to follow the functions used there
<neobrain> yes it encodes to RAM there, but I don't see where the XFBSource entry is created
<shuffle2> fail...forgot to build gcc45 for universal target...have to recompile EVERYTHING from macports
<BhaaL> why?
<BhaaL> you dont need gcc45 for universal
<shuffle2> i want gcc45
<shuffle2> :p
<BhaaL> universal means it will build a x86 and a x64 version of it
<shuffle2> just forgot to specify +universal
<BhaaL> and munge it together into one file
<shuffle2> yes and?
<BhaaL> why do you need an x64 version of it?
<BhaaL> :op
<BhaaL> :o
<shuffle2> just doing port install gcc45 doesn't give you a gcc able to build 32bit binaries
<BhaaL> it does
<shuffle2> well actually
<shuffle2> no it doesn't
<shuffle2> -dumpmachines didn't list 32bit target
<neobrain> ah.. const XFBSourceBase* const* xfbSourceList = FramebufferManager::GetXFBSource(xfbAddr, fbWidth, fbHeight, xfbCount);
<shuffle2> ...and now macports fails to build 64bit gmp wtf :x
<shuffle2> i need to nuke macports somehow :|
<n00b421> neobrain: managerbase...
<neobrain> hm?
<n00b421> declared in framebuffermanagerbase
<neobrain> I know o_o
<neobrain> I already know where it gets created now
<neobrain> erm
<neobrain> ...
<neobrain> In the line I posted
<n00b421> oh.. didn't recognized that... was just browsing and typing blindly...
<neobrain>
Quote:Quote:lol, afaik that has to do with the wii screen saver XD, but then again that's just what I always assumed
The Wii has a screen saver? News to me
EDIT: I don't play my Wii and leave it sitting there for long periods of time, so I've never seen the Wii screen saver in real life. You don't need to have a NAND dump or anything to do this on Dolphin, right? It should work with any Wii game?
afaik it just dims the screen
(01-22-2013, 04:06 AM)Shonumi Wrote: [ -> ]5) - Ignore Format Changes - Does what it says on the tin, i.e. ignores format changes to the EFB. I'm a bit curious as to how often format changes happen (and why); also, for confirmation, this option is always enabled on OpenGL, but it can be disabled on DX9 and DX11, correct?
The point of the emulation of format change emulation is that when the EFB format is being changed, the GC GPU doesn't perform any sort of automatic color conversion but just leaves the data as is. Thus, when changing from e.g. RGB565 to RGB888 the pixel data is just being reinterpreted for the new format.
I originally implemented this feature because it fixed an issue in Super Mario Sunshine. Turned out that the issue could be fixed differently (and easier) a few hundred revs later, but whatever. It still seems to have some sort of use because it fixes some issue in Skyward Sword.
I don't think any game particularly needs format change emulation, but it works around some limitations of what we can emulate in hw backends at all.
The option is always enabled on OpenGL because there's no support for format change emulation there.
Hey, thanks neobrain, fascinating reads so far; you're an excellent source as ever. I'll let you know when I'm finished.
Thanks too ED2, I feel quite stupid, seeing how obvious the Wii Screen Saver is :p
(01-22-2013, 04:06 AM)Shonumi Wrote: [ -> ]3) - DVD Root & Apploader - If you click on the Config button and go to the Paths tabs, you can specify paths for the DVD Root and Apploader. I just want to clarify what the DVD Root and the Apploader are. If you extract the files from an ISO, you get a directory structure that mirrors what's on the DVD you dumped, so Dolphin lets you specify the "root" directory of the game, effectively allowing you to boot games using these extracted files, correct? I have no idea what the apploader is. Useful for Homebrew on Dolphin or something?
My own experience in this point: when you right click the disc under Game Properties => Filesystem tab and choose extract all, you'll be prompted to choose a folder where the extracted content will be put plus a boot.dol and an apploader.img.
To load the game using the files from the directory you've just extracted, you need to set DVD Root as this folder and point apploader.img location in Dolphin settings. Then, booting the extracted boot.dol file will run the extracted copy of the game, as if you booted an ISO. So, this is my assumptions (maybe wrong, some dev please confirm this):
- Apploader.img should be something like a boot sector from a hard drive: it probably contain information about the "game type" (wii or GC), partition files and some other parameter needed by boot.dol
- boot.dol is the "app" that load the game files and make it run (something like the bootmgr, ntldr or grub loader from the various OSes)
With this process I was able to boot GC extracted games, never tried Wii games as they can have a lot of partitions (i.e. ISO dump of SSBB -- around 10-20 partitions) and the loading process maybe different. Furthermore, I found also that any setting specified in the game's INI is completely bypassed when you boot the same game from extracted files...
(01-22-2013, 04:06 AM)Shonumi Wrote: [ -> ]7) - Show Drives & Purge Cache - Misc. curiosity, under the View menu in the toolbar, what does the Show Drives option actually do? Purge Cache, I assume deletes the code cache, shader cache, both, or something else?
Purge cache will delete the content of the folder User\Cache, this folder is used to make a local cache of banner images and to avoid the need of extracting banner image from the ISOs every time you open Dolphin or refresh the list.
I may be wrong here, but if I remember correctly, "Show Drives" will check if a game backup disc is inserted in the computer's DVD drive and then add an entry in Game list corresponding to that game if there is a valid backup disc inserted in the DVD drive...