Dolphin, the GameCube and Wii emulator - Forums

Full Version: Understanding Dolphin: Wii Memory Architecture
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
(01-23-2015, 09:08 PM)Fiora Wrote: [ -> ]I think this is only true of the interpreter; the JIT just makes it a single write. I didn't actually know that a single write was the correct behavior though; I suppose the interpreter should be fixed.

I was going off what Delroth told me when I questioned why the FIFO write check in WriteToHardware only verified the page instead of the whole address. I have no idea what physical memory gets hit if you poke past the FIFO mmio, but even if it's mirrored through the whole page it will definitely miss the write-gather pipe and who knows what that will do.
Interesting, I had no idea that wasn't the intended hardware behavior. That sounds like violating that rule could be... interesting.
Most likely the write would just get spat out the other end of the FIFO like normal, the write-gather pipe is really only there to buffer things into nice 32-byte bursts.
(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]You might have a slightly easier time looking at https://github.com/magumagu/dolphin/blob/dynamic-bat/Source/Core/Core/PowerPC/MMU.cpp#L138 (from PR1882).  This is much closer to how a Wii actually works.

Ahhh. Is this the code that's used when a game is flagged as requiring Full MMU Emulation?

(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]MEM2 is 64MB starting at 0x10000000 on Wii.  Wii games usually map this using BATs to 0x90000000 and 0xD0000000.

I assume you mean 0x10000000 is the physical address of the start of MEM2 (so, 0x10000000-0x13ffffff)

(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]Whenever an exception is triggered on PowerPC, translation gets turned off, so games end up accessing 0x00000000 etc.

Ew, that's nasty.

(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]0x40000000, 0x70000000, and 0xE0000000 don't actually correspond to anything on hardware.  Some Gamecube games set up the segment registers/page tables to map 0x40000000 or 0x70000000; Dolphin has a hack (which is turned on by disabling the "MMU" setting) that backs these with actual memory because it didn't have a decent MMU implementation for a long time.

Games use the locked L1 extension allocate cache lines out of the L1 cache; as far as we know, all games use 0xE0000000 for this, but that isn't required by the hardware.  (Games that do this modify the BAT to map 0xE0000000 to 0xE0000000.)  
Is that right? You listed the same address for both.

(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]You can probably find the actual Broadway manual on the web, which has a complete description of how this works.
Well, the Broadway was essentially custom-built for Nintendo, so there's no official datasheet.
However, I was able to track down a leaked copy of a preliminary manual for it. If people want me to host a copy somewhere, just let me know; when I'm done, I want to compile everything I've learned and put it on the wiki.

Looking at the description of the "Locked L1 Cache", it *looks* like it's simply a way to force certain (presumably oft-allocated) physical addresses to be kept in the cache; there doesn't seem to be an actual physical address, so the BAT wouldn't work.
However, apparently you can DMA between the locked cache and main memory; perhaps the games use paging in a manner similar to ARAM (as tueidj describes)?


(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]And yes, the "(em_address & 0xC8000000) == 0xC8000000" thing is a bug.
Hah! The moment I read that line I was *very* suspicious. (And that's why I'm asking all these questions.)

(01-23-2015, 03:15 PM)magumagu Wrote: [ -> ]Oh, and one more thing: the Wii has some extra BAT registers which you might have missed: https://github.com/dolphin-emu/dolphin/blob/96a2b74c02e3ee12b23b9a2fcfcf51c32d9524ed/Source/Core/Core/PowerPC/Gekko.h#L780 .

Oooh, I see those in the Broadway manual; there's not the slightest suggestion of them in the generic PPC manual, nor the 750 manual.
Thanks!

(01-23-2015, 07:57 PM)tueidj Wrote: [ -> ]
Quote:If it can't be accessed directly by the CPU, then how can segment descriptors help a game access ARAM?
By using paging. A small area of MEM1 is used to temporarily hold a few pages of ARAM, when a DSI/page fault occurs DMA transfers are used to swap pages in and out.


Okay, if ARAM stands for "Audio RAM" (as opposed to VRAM for Video RAM), then I'm guessing that using ARAM in this way is only useful for *reading* from ARAM (because you can easily cause reads to 'fault in' data from ARAM->EXTMEM, but automatically pushing EXTMEM->ARAM is essentially impossible.)


Okay, so my current understanding is this:
* GC games start out being able to access physical addresses 0x00000000-0x0fffffff (regardless of whether they map to a real device or not)
via both 0x800000000-0x8fffffff (cached, write-back, good for real RAM) and 0xc0000000-0xcfffffff (non-cached, write-through, good for MMIO devices).
* Wii games probably start out the same way. (I'm not at home right now, so I can't see what BATs 4-7 look like, although I suspect that they start out disabled; 100% GC compatibility might be challenging otherwise.)

And *nothing else* is available. For all other prefixes -- 0x7, 0x4, 0xD, 0x9, 0x1, 0xE, etc. every single game must contain code to configure the unused BATs (2-3 for GC, 2-7 for Wii) and/or set up segments and page tables for everything else. (In fact, once it's got control, a game is probably even free to change BATs 0 and 1, but there is little or no benefit to doing this, so I expect all games leave [DI]BAT[01][LU] untouched.)

I'm guessing that, despite this, the vast majority of games share a common virtual memory layout (I remember seeing a comment somewhere about Nintendo having recommendations on MMU management). For these games, Dolphin essentially acts as if there were BAT mappings for the common segments, which is faster than consulting the page tables. Only if a game touches an atypical memory address does {ReadFrom,WriteTo}Hardware actually consult the page tables (the code way at the end).

(01-23-2015, 07:57 PM)tueidj Wrote: [ -> ]Dolphin has lots of issues because it's only designed to run games, which don't really do very interesting stuff with memory configurations. As an example, Star Wars: Clone Wars does use a custom setup and currently doesn't run.

Is this a GC or Wii game? What's the UPC code? Amazon or somesuch link?

I'd like to assemble a collection of games that covers most of the various configurations (i.e. a game that uses 0x7, a game that uses 0x4, a game that uses 0xD, a game that uses 0xE, a game that pages in from ARAM, etc.)
The only game I currently own is Lego Indiana Jones: The Original Adventures; I used it and Indiana Pwns to jailbreak my Wii and install the Homebrew channel so I could poke at the hardware registers. Does anyone have a list of games that cover these various eventualities?

I've got some ideas on ways to rework the MMU emulation to be more efficient, while supporting a greater number of atypical games. However, I need a set of typical and non-typical games in order to check for regressions.
If you'd like to work on the code you should probably drop by #dolphin-dev on Freenode IRC; the latency for questions on the MMU code and hardware behavior is probably a lot lower there.
(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]You might have a slightly easier time looking at https://github.com/magumagu/dolphin/blob/dynamic-bat/Source/Core/Core/PowerPC/MMU.cpp#L138 (from PR1882).  This is much closer to how a Wii actually works.

Ahhh.  Is this the code that's used when a game is flagged as requiring Full MMU Emulation?

This is from a work-in-progress branch; "Full MMU emulation" in dolphin is basically just turning on emulation of DSI exceptions.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]MEM2 is 64MB starting at 0x10000000 on Wii.  Wii games usually map this using BATs to 0x90000000 and 0xD0000000.

I assume you mean 0x10000000 is the physical address of the start of MEM2 (so, 0x10000000-0x13ffffff)

Yes.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]Whenever an exception is triggered on PowerPC, translation gets turned off, so games end up accessing 0x00000000 etc.

Ew, that's nasty.

This is how every computer in existence works?

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 03:04 PM)magumagu Wrote: [ -> ]0x40000000, 0x70000000, and 0xE0000000 don't actually correspond to anything on hardware.  Some Gamecube games set up the segment registers/page tables to map 0x40000000 or 0x70000000; Dolphin has a hack (which is turned on by disabling the "MMU" setting) that backs these with actual memory because it didn't have a decent MMU implementation for a long time.

Games use the locked L1 extension allocate cache lines out of the L1 cache; as far as we know, all games use 0xE0000000 for this, but that isn't required by the hardware.  (Games that do this modify the BAT to map 0xE0000000 to 0xE0000000.)  
Is that right? You listed the same address for both.

Yes, that's right.

[quote pid='355908' dateline='1422045313']
Looking at the description of the "Locked L1 Cache", it *looks* like it's simply a way to force certain (presumably oft-allocated) physical addresses to be kept in the cache; there doesn't seem to be an actual physical address, so the BAT wouldn't work.
However, apparently you can DMA between the locked cache and main memory; perhaps the games use paging in a manner similar to ARAM (as tueidj describes)?
[/quote]

Locked L1 is primarily used for video codecs, to quickly move video data between the CPU and memory.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 07:57 PM)tueidj Wrote: [ -> ]
Quote:If it can't be accessed directly by the CPU, then how can segment descriptors help a game access ARAM?
By using paging. A small area of MEM1 is used to temporarily hold a few pages of ARAM, when a DSI/page fault occurs DMA transfers are used to swap pages in and out.

Okay, if ARAM stands for "Audio RAM" (as opposed to VRAM for Video RAM), then I'm guessing that using ARAM in this way is only useful for *reading* from ARAM (because you can easily cause reads to 'fault in' data from ARAM->EXTMEM, but automatically pushing EXTMEM->ARAM is essentially impossible.)

Well, you can page out data to ARAM to make room to page in data from ARAM, sort of like a swap file. There's a library a bunch of Gamecube games use which manages this automatically.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]Okay, so my current understanding is this:
* GC games start out being able to access physical addresses 0x00000000-0x0fffffff (regardless of whether they map to a real device or not)
via both 0x800000000-0x8fffffff (cached, write-back, good for real RAM) and 0xc0000000-0xcfffffff (non-cached, write-through, good for MMIO devices).
* Wii games probably start out the same way. (I'm not at home right now, so I can't see what BATs 4-7 look like, although I suspect that they start out disabled; 100% GC compatibility might be challenging otherwise.)

Wii games actually start out with a mappings for 0x90000000/0xD0000000.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]And *nothing else* is available. For all other prefixes -- 0x7, 0x4, 0xD, 0x9, 0x1, 0xE, etc. every single game must contain code to configure the unused BATs (2-3 for GC, 2-7 for Wii) and/or set up segments and page tables for everything else.  (In fact, once it's got control, a game is probably even free to change BATs 0 and 1, but there is little or no benefit to doing this, so I expect all games leave [DI]BAT[01][LU] untouched.)

I'm guessing that, despite this, the vast majority of games share a common virtual memory layout (I remember seeing a comment somewhere about Nintendo having recommendations on MMU management).  For these games, Dolphin essentially acts as if there were BAT mappings for the common segments, which is faster than consulting the page tables.  Only if a game touches an atypical memory address does {ReadFrom,WriteTo}Hardware actually consult the page tables (the code way at the end).

This, exactly.

(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]
(01-23-2015, 07:57 PM)tueidj Wrote: [ -> ]Dolphin has lots of issues because it's only designed to run games, which don't really do very interesting stuff with memory configurations. As an example, Star Wars: Clone Wars does use a custom setup and currently doesn't run.

Is this a GC or Wii game? What's the UPC code? Amazon or somesuch link?

https://wiki.dolphin-emu.org/index.php?title=Star_Wars:_The_Clone_Wars

https://wiki.dolphin-emu.org/index.php?title=Star_Wars_Rogue_Squadron_III:_Rebel_Strike does some sort of unusual stuff.

If you want to have fun with BAT stuff, https://wiki.dolphin-emu.org/index.php?title=Cars_2 ; among other things, this uses invalid BAT register settings.
(01-24-2015, 06:35 AM)Stevie-O Wrote: [ -> ]Looking at the description of the "Locked L1 Cache", it *looks* like it's simply a way to force certain (presumably oft-allocated) physical addresses to be kept in the cache; there doesn't seem to be an actual physical address, so the BAT wouldn't work.
However, apparently you can DMA between the locked cache and main memory; perhaps the games use paging in a manner similar to ARAM (as tueidj describes)?
The locked cache doesn't cache physical memory; it actually causes a machine check if it tries. In a sense it caches non-existent memory, which works because it never gets evicted from the cache.

Quote:Okay, if ARAM stands for "Audio RAM" (as opposed to VRAM for Video RAM), then I'm guessing that using ARAM in this way is only useful for *reading* from ARAM (because you can easily cause reads to 'fault in' data from ARAM->EXTMEM, but automatically pushing EXTMEM->ARAM is essentially impossible.)
A write can just as easily trigger a DSI/page fault.
Pages get pushed out to ARAM when the space they currently occupy in main memory is required for another page. The tricky part is finding a good algorithm to decide which pages get replaced in order to minimize the number of page faults.

Quote:Okay, so my current understanding is this:
* GC games start out being able to access physical addresses 0x00000000-0x0fffffff (regardless of whether they map to a real device or not)
via both 0x800000000-0x8fffffff (cached, write-back, good for real RAM) and 0xc0000000-0xcfffffff (non-cached, write-through, good for MMIO devices).
* Wii games probably start out the same way. (I'm not at home right now, so I can't see what BATs 4-7 look like, although I suspect that they start out disabled; 100% GC compatibility might be challenging otherwise.)
The wii maps MEM2 by default. It only disables the secondary BATs when it needs to switch to gamecube mode (or when the powerpc is reset).

Quote:And *nothing else* is available. For all other prefixes -- 0x7, 0x4, 0xD, 0x9, 0x1, 0xE, etc. every single game must contain code to configure the unused BATs (2-3 for GC, 2-7 for Wii) and/or set up segments and page tables for everything else.  (In fact, once it's got control, a game is probably even free to change BATs 0 and 1, but there is little or no benefit to doing this, so I expect all games leave [DI]BAT[01][LU] untouched.)
A game can always disable translation (switch to real mode) for whatever reason, meaning the physical ranges are accessible. This happens every time an exception occurs. When in real mode all addresses are valid; a lot of them may just return junk.

Quote:Is this a GC or Wii game? What's the UPC code? Amazon or somesuch link?
What do I look like, Google?
Quote:I'd like to assemble a collection of games that covers most of the various configurations (i.e. a game that uses 0x7, a game that uses 0x4, a game that uses 0xD, a game that uses 0xE, a game that pages in from ARAM, etc.)  
The only game I currently own is Lego Indiana Jones: The Original Adventures; I used it and Indiana Pwns to jailbreak my Wii and install the Homebrew channel so I could poke at the hardware registers.  Does anyone have a list of games that cover these various eventualities?
Random games (both wii and gamecube) use the 0xExxxxxxx/locked cache region, depending on if the devs thought it was beneficial. One in particular that sticks out in my mind is "Carmen Sandiego: The Secret of the Stolen Drums" because it has a programming mistake that causes frequent machine check exceptions.
Not sure there's any wii games that use 0xDxxxxxxx; it's just uncached MEM2 anyway, you can poke it with homebrew.
For everything else you want gamecube games:
- Nintendo's "official" VM SDK uses the 0x7E000000-0x7FFFFFFF region. Dozens of games use this, including some that don't even need it. Shrek 2, Medal of Honor, Jedi Knight 2, Prince of Persia...
- The Star Wars Rogue Squadron games have their own paging implementation that uses the same region
- Timesplitters Future Perfect and Second Sight have their own paging implementation that uses the 0xAxxxxxxx range.
- Spider-man 2 and Ultimate Spider-man have their own paging implementation that uses 0x7xxxxxxx and 0xFxxxxxxx ranges.
- F-Zero GX decides to alias a few select pages to somewhere in the 0x2xxxxxxx range.
- Star Wars: Clone Wars resizes the MEM1 BATs and uses the newly unallocated portion for paging.
- "Seaworld Adventure Park: Shamu's Deep Sea Adventures" (yes this is a real game) apparently has its own custom paging implementation. Haven't looked at it.
Cars 2 and Disney Infinity are both games that make heavy abuse of MMU features to the point where they aren't fully understood; last I heard Cars 2 was confirmed to set overlapping BATs on boot (something that the Broadway manual says will cause undefined behavior). Part of the goal of magumagu's branch is to get those working (eventually), so they're probably good things to poke at.
Pages: 1 2