• Login
  • Register
  • Dolphin Forums
  • Home
  • FAQ
  • Download
  • Wiki
  • Code


Dolphin, the GameCube and Wii emulator - Forums › Dolphin Emulator Discussion and Support › Development Discussion v
« Previous 1 ... 46 47 48 49 50 ... 114 Next »

New AX HLE: what is it and how does it work?
View New Posts | View Today's Posts

Pages (10): 1 2 3 4 5 ... 10 Next »
Jump to page 
Thread Rating:
  • 6 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Modes
New AX HLE: what is it and how does it work?
11-16-2012, 06:17 PM (This post was last modified: 12-04-2012, 05:03 AM by delroth.)
#1
delroth Offline
Making the world a better place through reverse engineered DSP firmwares
**********
Developers (Some Administrators and Super Moderators)
Posts: 1,354
Threads: 63
Joined: Aug 2011
A month ago, NaturalViolence told me it was kind of sad Dolphin devs didn't write a lot about how emulation works and how they try to improve it. Recently I've been working on a rewrite of the old DSP high-level emulation (HLE). Few people actually know about how sound emulation works in Dolphin, even among the developers, so this is an interesting topic to explain in detail IMO. I'll mostly talk about GC here, but Wii sound handling works almost the same so most of this post should apply to Wii as well.

An edited version of this article is available here: http://blog.lse.epita.fr/articles/38-emulating-the-gamecube-audio-processing-in-dolphin.html

This is a three parts article:
Part 1: how sound is processed in a Gamecube
Part 2: AX features and DSP/CPU communication
Part 3: AX HLE in Dolphin, previous vs. new

Part 1: how sound is processed in a Gamecube

There are three main internal components related to sound in a Gamecube: the ARAM, the AI and the DSP:
  • ARAM is an auxiliary memory which is used to store sound data. The CPU cannot access ARAM directly, it can only read/write blocks of data from RAM to ARAM (or from ARAM to RAM) using DMA requests. As ARAM is quite large, games often use it to store more than sound data: for example, WWE Day of Reckoning 2 uses it to store animation data (and a bug in DMA handling causes a crash because the data it writes is corrupted).
  • The AI (Audio Interface) is responsible for getting sound data from RAM and sending it to your TV. It performs an optional sample rate conversion (32KHz -> 48KHz) and converts the data to an analog signal that is sent through the cables to your audio output device. The input data is read at a regular interval from RAM (not ARAM), usually every 0.25ms 32 bytes of input data is read (each sound sample is 2 bytes, so 32 bytes is 16 sound samples, which is 8 stereo sound samples, and 8 samples every 0.25ms == 32KHz sound).
  • The DSP is what processes all the sounds a game wants to play and outputs a single stereo stream. Its job is to perform volume changes on the sounds, sample rate conversion (converting 4KHz sounds which take less space to 32KHz sounds - this is needed because you can't mix together sounds that are not the same rate). It can optionally do a lot of other stuff with the sounds (delaying to simulate 3D sound, filtering, handling surround sound, etc.).

ARAM and AI are not that hard to emulate: once you understand how they work, they are both simple chips which can only perform one function and don't communicate a lot with the CPU. You just need to have a precise enough timing for AI emulation, and everything is fine.

DSP is a lot harder to emulate properly, for two reasons I have not mentioned yet. First, it is a programmable CPU. All the mixing, filtering, etc. are part of a program that is sent to the DSP by the game, and the DSP behavior varies depending on the program it receives. For example, the DSP is not only used for sound processing, but also to unlock memory cards, and to crypt/decrypt data sent to a GBA using the official link cable. Even for sound processing, not every game uses the same DSP code. The second reason is that it can communicate with the main Gamecube CPU, read RAM and ARAM and write to RAM. This allows games to use a complicated communication protocol between CPU and DSP.

We call a program running on the DSP a UCode ("microcode"). Because the DSP is programmable, it would seem like the only way to emulate it properly is to use low level emulation: running instructions one by one from a program to reproduce accurately what the real DSP does. However, while it is programmable, there are actually very few different UCodes used by games. On Gamecube, there are only 3 UCodes we know of: the AX UCode (used in most games because it is distributed with Nintendo's SDK), the Zelda UCode (called that way because it's used in Zelda games, but it is also used for some Mario games and some other first party games), and the JAC UCode (early version of the Zelda UCode, used in the Gamecube IPL/BIOS as well as Luigi's Mansion). That means if we can reproduce the behavior of these three UCodes, we can emulate the audio processing in most games without having to emulate the DSP instructions.

I started working on AX HLE last week because I want to play Skies of Arcadia Legends and Tales of Symphonia, which are two games that had completely broken audio with the previous AX HLE implementation. I added a new hack to "fix" the bug that caused bad music emulation, but fixing this made me even more interested in rewriting the whole thing to make it cleaner. I wasn't following Dolphin development when the current AX HLE was developed. However, it looks to me as if it was written without actually looking at the DSP code, only looking at what is sent to the DSP and what comes out. I don't know if people at the time had the capability to disassemble DSP code, but it is a very bad way to emulate AX anyway: some parts of the emulation code are completely WTF, and the more you understand how AX works the less you understand how the current AX HLE emulation was able to work and output sound in most cases. That's why, 3 days ago I decided I should start from scratch and re-implement AX HLE.

I'm not completely done with it yet: there are still a few bugs and desyncs, which are mostly identified. Still, the current new-ax-hle audio is largely better than the audio on latest master revisions. Small comparison for old HLE, new HLE and LLE on Tales of Symphonia main menu.

In a next article I'll talk about how audio is emulated in Dolphin, how AX works internally and why the previous AX implementation shouldn't actually be working that well Tongue
Pierre "delroth" Bourdon - @delroth_ - Blog

<@neobrain> that looks sophisticated enough to not be a totally dumb thing to do
Website Find
Reply
11-16-2012, 10:06 PM
#2
neobrain Offline
"Wow, I made my code 1000x faster! That means I can make it 2048x slower now!"
**********
Developers (Some Administrators and Super Moderators)
Posts: 3,209
Threads: 50
Joined: Jun 2009
Nice one, keep it coming :>
My blog
Me on Twitter
My wishlist on Amazon.de
Find
Reply
11-16-2012, 10:20 PM
#3
DefenderX Offline
The comedy never ends.
*******
Posts: 4,617
Threads: 186
Joined: Dec 2011
nice explanation delroth Smile
[Image: IEZtsj]
[Image: 8fhx2zum.gif]
Website Find
Reply
11-16-2012, 11:31 PM
#4
Zee530 Offline
Above and Beyond
*******
Posts: 1,747
Threads: 12
Joined: Jan 2011
So you only fix bugs and implement changes when you want to play a game that has issues? if only we could all be like you Smile
......?????
Find
Reply
11-17-2012, 12:19 AM
#5
delroth Offline
Making the world a better place through reverse engineered DSP firmwares
**********
Developers (Some Administrators and Super Moderators)
Posts: 1,354
Threads: 63
Joined: Aug 2011
(11-16-2012, 11:31 PM)Zee530 Wrote: So you only fix bugs and implement changes when you want to play a game that has issues? if only we could all be like you Smile
Well, I started contributing to Dolphin because I noticed a bug when I was playing Baten Kaitos and thought I should try to fix it Tongue .

I also work on some issues just for the technical challenge of it: WWEBig GrinOR2 ARAM stuff for example (not interested in the game at all). But obviously I'm more interested in fixing stuff in games I like and I enjoy playing (or would like to play).
Pierre "delroth" Bourdon - @delroth_ - Blog

<@neobrain> that looks sophisticated enough to not be a totally dumb thing to do
Website Find
Reply
11-17-2012, 01:27 AM
#6
Garteal Offline
「Lab Mem. 004」
********
Global Moderators
Posts: 2,095
Threads: 24
Joined: Aug 2011
Thank you for the explanation and initiative to rewrite AX HLE from scratch delroth.
So far I'm able to follow the article, so I'm looking forward to the next part.

Now NV needs to convince neobrain to start writing more stuff. Tongue
Find
Reply
11-17-2012, 01:28 AM
#7
delroth Offline
Making the world a better place through reverse engineered DSP firmwares
**********
Developers (Some Administrators and Super Moderators)
Posts: 1,354
Threads: 63
Joined: Aug 2011
Part 2: how AX works

AX is a low-level audio library for Gamecube games, which comes with a builtin UCode to perform audio signal processing on the DSP. I'll first talk about what it can do, then explain how the UCode knows what it should do.

Luckily, Nintendo gives us a lot of informations about the role of the DSP in a patent filed on Aug 23 2000: US7369665, "Method and apparatus for mixing sound signals". Figures 8, 9A and 9B are especially interesting in our case because they describe precisely what the DSP does internally and how inputs and outputs interact with each other. That helps, but most of this information could already be discovered by reverse engineering the UCode anyway (I learned the existence of this patent pretty late).

The basic role of the DSP is to get several sounds and mix them together to give a single sound. The sounds that it has to mix are provided through a list of Parameter Blocks (PB). Each PB corresponds to a sound to be mixed. It contains where to find the input sound data, but also a lot of configuration options: input sample rate, sound volume, where it should be mixed and at what volume (left channel/right channel/surround), if the sounds loop, from where does the loop start, etc.

Every 5ms AX gets a list of PB and mixes each PB to 3 channels: Left, Right and Surround. It then sends 5ms of output to the RAM, at an address provided by the CPU. Sometimes being able to change sound data only every 5ms is not enough: to overcome that, each PB has a list of updates to be applied every millisecond. This allows sub-5ms granularity in sound mixing configuration. AX also provides a way to add audio effects on the L/R/S streams through the use of AUX channels. Each PB can be mixed to L/R/S but also to AUXA L/R/S and AUXB L/R/S. Then, the CPU can ask to get the contents of the AUXA and AUXB mixing buffers, replace them with its own data, and ask the DSP to mix AUXA and AUXB with the main L/R/S channels.

That's about it for the main features of AX. Some more things can be done optionally (for example Initial Time Delay, used to delay one channel to simulate 3D sound) but they are not used that often by games. Let's see how the CPU sends commands to the DSP.

The DSP has two ways to communicate with the game: through DMA, which allows it to read or write to RAM at any address it wants, and through mailbox registers, which is a more synchronous way to exchange small amounts of data (32 bits at a time) with the CPU. Usually, mailbox registers are used for synchronization and simple commands. For more complicated commands the CPU sends an address to the DSP via mailbox, and the DSP gets the data at this address through DMA.

With AX, about the only thing received through mailboxes (excluding UCode switching stuff which is not relevant to sound processing) is an address to a larger block of data which contains commands for the DSP. Here is the list of commands that AX understands and that I have reverse engineered:
  • Command 00: SETUP, initializes internal mixing buffers with a constant value or a value and a delta. Usually just initializes to 0.
  • Command 02: PBADDR, gives the DSP the address in RAM of the first PB. Each PB contains the address of the next PB, so knowing only the address of the first PB is enough to get the whole list.
  • Command 03: PROCESS, does all the audio processing and mixes the PBs to internal buffers.
  • Command 04: MIX_AUXA, sends the contents of the AUXA buffers to the CPU, receives processed AUXA, and mix it with the main channels.
  • Command 05: MIX_AUXB, same as MIX_AUXA for AUXB
  • Command 06: UPLOAD_LRS, sends the contents of the main L/R/S channels to the CPU.
  • Command 0D: MORE, read more commands from RAM and start executing them. I suspect this is used for long command lists, but I've never seen it used.
  • Command 0E: OUTPUT, interlaces L/R channel, clamp to 16 bits and send to RAM, where it will most likely get picked up by the Audio Interface.
  • Command 0F: END, signals the end of a command list.

A few more commands exist, but handling these commands is enough to run most games I've found. Actually, only handling PBADDR, PROCESS, OUTPUT and END should be enough.

When AX is done with a command list, it sends an interrupt to the CPU to signal that it is ready to receive more data. This is very important because it is the only way for the CPU to know that the data it requested to be uploaded from the DSP is actually valid and done copying/processing.

I think this is about it for the AX explanations. As you may notice, it does not do a lot of things: once you know how it works, reproducing its behavior in an emulator is actually pretty easy. In the next part of this article I'll talk about how we previously implemented AX HLE and how we do it now, so you can see why this branch actually fixes so much issues Smile .
Pierre "delroth" Bourdon - @delroth_ - Blog

<@neobrain> that looks sophisticated enough to not be a totally dumb thing to do
Website Find
Reply
11-17-2012, 01:50 AM
#8
LordVador Offline
Christmas Vader
*******
Posts: 8,852
Threads: 1,908
Joined: Mar 2011
What level of improvement can we expect from this rewrite?
[color=#ff0000][color=#006600]i5 3570K @ 4.5GHz/GTX 660 Ti/RAM 4GB/Win7 x64[/color][/color]
Find
Reply
11-17-2012, 02:06 AM
#9
delroth Offline
Making the world a better place through reverse engineered DSP firmwares
**********
Developers (Some Administrators and Super Moderators)
Posts: 1,354
Threads: 63
Joined: Aug 2011
Look at the comments here: http://code.google.com/p/dolphin-emu/source/detail?r=5a2d8661d7113ed2fd1c51d5f4e10b5ce385459a&name=new-ax-hle

Basically, when I'll be done with AX, AXWii, Zelda, ZeldaWii and JAC, HLE should be as good as LLE on 99.9% of games. At the moment there are still some bugs with AX though, so don't expect that work to be done before 2013.
Pierre "delroth" Bourdon - @delroth_ - Blog

<@neobrain> that looks sophisticated enough to not be a totally dumb thing to do
Website Find
Reply
11-17-2012, 02:11 AM
#10
Duke Nukem Offline
Banned
Posts: 1,724
Threads: 48
Joined: Mar 2011
Holy shit this is awesome news, I think I'm gonna wet my pants. 2013 is gonna be a sweet Dolphin year.
Find
Reply
« Next Oldest | Next Newest »
Pages (10): 1 2 3 4 5 ... 10 Next »
Jump to page 


  • View a Printable Version
  • Subscribe to this thread
Forum Jump:


Users browsing this thread: 1 Guest(s)



Powered By MyBB | Theme by Fragma

Linear Mode
Threaded Mode