New AX HLE: what is it and how does it work?

[delroth]

A month ago, NaturalViolence told me it was kind of sad Dolphin devs didn't write a lot about how emulation works and how they try to improve it. Recently I've been working on a rewrite of the old DSP high-level emulation (HLE). Few people actually know about how sound emulation works in Dolphin, even among the developers, so this is an interesting topic to explain in detail IMO. I'll mostly talk about GC here, but Wii sound handling works almost the same so most of this post should apply to Wii as well.

An edited version of this article is available here: http://blog.lse.epita.fr/articles/38-emulating-the-gamecube-audio-processing-in-dolphin.html

This is a three parts article:
Part 1: how sound is processed in a Gamecube
Part 2: AX features and DSP/CPU communication
Part 3: AX HLE in Dolphin, previous vs. new

Part 1: how sound is processed in a Gamecube

There are three main internal components related to sound in a Gamecube: the ARAM, the AI and the DSP:

• ARAM is an auxiliary memory which is used to store sound data. The CPU cannot access ARAM directly, it can only read/write blocks of data from RAM to ARAM (or from ARAM to RAM) using DMA requests. As ARAM is quite large, games often use it to store more than sound data: for example, WWE Day of Reckoning 2 uses it to store animation data (and a bug in DMA handling causes a crash because the data it writes is corrupted).
• The AI (Audio Interface) is responsible for getting sound data from RAM and sending it to your TV. It performs an optional sample rate conversion (32KHz -> 48KHz) and converts the data to an analog signal that is sent through the cables to your audio output device. The input data is read at a regular interval from RAM (not ARAM), usually every 0.25ms 32 bytes of input data is read (each sound sample is 2 bytes, so 32 bytes is 16 sound samples, which is 8 stereo sound samples, and 8 samples every 0.25ms == 32KHz sound).
• The DSP is what processes all the sounds a game wants to play and outputs a single stereo stream. Its job is to perform volume changes on the sounds, sample rate conversion (converting 4KHz sounds which take less space to 32KHz sounds - this is needed because you can't mix together sounds that are not the same rate). It can optionally do a lot of other stuff with the sounds (delaying to simulate 3D sound, filtering, handling surround sound, etc.).

ARAM and AI are not that hard to emulate: once you understand how they work, they are both simple chips which can only perform one function and don't communicate a lot with the CPU. You just need to have a precise enough timing for AI emulation, and everything is fine.

DSP is a lot harder to emulate properly, for two reasons I have not mentioned yet. First, it is a programmable CPU. All the mixing, filtering, etc. are part of a program that is sent to the DSP by the game, and the DSP behavior varies depending on the program it receives. For example, the DSP is not only used for sound processing, but also to unlock memory cards, and to crypt/decrypt data sent to a GBA using the official link cable. Even for sound processing, not every game uses the same DSP code. The second reason is that it can communicate with the main Gamecube CPU, read RAM and ARAM and write to RAM. This allows games to use a complicated communication protocol between CPU and DSP.

We call a program running on the DSP a UCode ("microcode"). Because the DSP is programmable, it would seem like the only way to emulate it properly is to use low level emulation: running instructions one by one from a program to reproduce accurately what the real DSP does. However, while it is programmable, there are actually very few different UCodes used by games. On Gamecube, there are only 3 UCodes we know of: the AX UCode (used in most games because it is distributed with Nintendo's SDK), the Zelda UCode (called that way because it's used in Zelda games, but it is also used for some Mario games and some other first party games), and the JAC UCode (early version of the Zelda UCode, used in the Gamecube IPL/BIOS as well as Luigi's Mansion). That means if we can reproduce the behavior of these three UCodes, we can emulate the audio processing in most games without having to emulate the DSP instructions.

I started working on AX HLE last week because I want to play Skies of Arcadia Legends and Tales of Symphonia, which are two games that had completely broken audio with the previous AX HLE implementation. I added a new hack to "fix" the bug that caused bad music emulation, but fixing this made me even more interested in rewriting the whole thing to make it cleaner. I wasn't following Dolphin development when the current AX HLE was developed. However, it looks to me as if it was written without actually looking at the DSP code, only looking at what is sent to the DSP and what comes out. I don't know if people at the time had the capability to disassemble DSP code, but it is a very bad way to emulate AX anyway: some parts of the emulation code are completely WTF, and the more you understand how AX works the less you understand how the current AX HLE emulation was able to work and output sound in most cases. That's why, 3 days ago I decided I should start from scratch and re-implement AX HLE.

I'm not completely done with it yet: there are still a few bugs and desyncs, which are mostly identified. Still, the current new-ax-hle audio is largely better than the audio on latest master revisions. Small comparison for old HLE, new HLE and LLE on Tales of Symphonia main menu.

In a next article I'll talk about how audio is emulated in Dolphin, how AX works internally and why the previous AX implementation shouldn't actually be working that well

[neobrain]

Nice one, keep it coming :>

[DefenderX]

nice explanation delroth

[Zee530]

So you only fix bugs and implement changes when you want to play a game that has issues? if only we could all be like you

[delroth]

(11-16-2012, 11:31 PM)Zee530 Wrote: So you only fix bugs and implement changes when you want to play a game that has issues? if only we could all be like you

Well, I started contributing to Dolphin because I noticed a bug when I was playing Baten Kaitos and thought I should try to fix it .

I also work on some issues just for the technical challenge of it: WWEOR2 ARAM stuff for example (not interested in the game at all). But obviously I'm more interested in fixing stuff in games I like and I enjoy playing (or would like to play).

[Garteal]

Thank you for the explanation and initiative to rewrite AX HLE from scratch delroth.
So far I'm able to follow the article, so I'm looking forward to the next part.

Now NV needs to convince neobrain to start writing more stuff.

[delroth]

Part 2: how AX works

AX is a low-level audio library for Gamecube games, which comes with a builtin UCode to perform audio signal processing on the DSP. I'll first talk about what it can do, then explain how the UCode knows what it should do.

Luckily, Nintendo gives us a lot of informations about the role of the DSP in a patent filed on Aug 23 2000: US7369665, "Method and apparatus for mixing sound signals". Figures 8, 9A and 9B are especially interesting in our case because they describe precisely what the DSP does internally and how inputs and outputs interact with each other. That helps, but most of this information could already be discovered by reverse engineering the UCode anyway (I learned the existence of this patent pretty late).

The basic role of the DSP is to get several sounds and mix them together to give a single sound. The sounds that it has to mix are provided through a list of Parameter Blocks (PB). Each PB corresponds to a sound to be mixed. It contains where to find the input sound data, but also a lot of configuration options: input sample rate, sound volume, where it should be mixed and at what volume (left channel/right channel/surround), if the sounds loop, from where does the loop start, etc.

Every 5ms AX gets a list of PB and mixes each PB to 3 channels: Left, Right and Surround. It then sends 5ms of output to the RAM, at an address provided by the CPU. Sometimes being able to change sound data only every 5ms is not enough: to overcome that, each PB has a list of updates to be applied every millisecond. This allows sub-5ms granularity in sound mixing configuration. AX also provides a way to add audio effects on the L/R/S streams through the use of AUX channels. Each PB can be mixed to L/R/S but also to AUXA L/R/S and AUXB L/R/S. Then, the CPU can ask to get the contents of the AUXA and AUXB mixing buffers, replace them with its own data, and ask the DSP to mix AUXA and AUXB with the main L/R/S channels.

That's about it for the main features of AX. Some more things can be done optionally (for example Initial Time Delay, used to delay one channel to simulate 3D sound) but they are not used that often by games. Let's see how the CPU sends commands to the DSP.

The DSP has two ways to communicate with the game: through DMA, which allows it to read or write to RAM at any address it wants, and through mailbox registers, which is a more synchronous way to exchange small amounts of data (32 bits at a time) with the CPU. Usually, mailbox registers are used for synchronization and simple commands. For more complicated commands the CPU sends an address to the DSP via mailbox, and the DSP gets the data at this address through DMA.

With AX, about the only thing received through mailboxes (excluding UCode switching stuff which is not relevant to sound processing) is an address to a larger block of data which contains commands for the DSP. Here is the list of commands that AX understands and that I have reverse engineered:

• Command 00: SETUP, initializes internal mixing buffers with a constant value or a value and a delta. Usually just initializes to 0.
• Command 02: PBADDR, gives the DSP the address in RAM of the first PB. Each PB contains the address of the next PB, so knowing only the address of the first PB is enough to get the whole list.
• Command 03: PROCESS, does all the audio processing and mixes the PBs to internal buffers.
• Command 04: MIX_AUXA, sends the contents of the AUXA buffers to the CPU, receives processed AUXA, and mix it with the main channels.
• Command 05: MIX_AUXB, same as MIX_AUXA for AUXB
• Command 06: UPLOAD_LRS, sends the contents of the main L/R/S channels to the CPU.
• Command 0D: MORE, read more commands from RAM and start executing them. I suspect this is used for long command lists, but I've never seen it used.
• Command 0E: OUTPUT, interlaces L/R channel, clamp to 16 bits and send to RAM, where it will most likely get picked up by the Audio Interface.
• Command 0F: END, signals the end of a command list.

A few more commands exist, but handling these commands is enough to run most games I've found. Actually, only handling PBADDR, PROCESS, OUTPUT and END should be enough.

When AX is done with a command list, it sends an interrupt to the CPU to signal that it is ready to receive more data. This is very important because it is the only way for the CPU to know that the data it requested to be uploaded from the DSP is actually valid and done copying/processing.

I think this is about it for the AX explanations. As you may notice, it does not do a lot of things: once you know how it works, reproducing its behavior in an emulator is actually pretty easy. In the next part of this article I'll talk about how we previously implemented AX HLE and how we do it now, so you can see why this branch actually fixes so much issues .

[LordVador]

What level of improvement can we expect from this rewrite?

[delroth]

Look at the comments here: http://code.google.com/p/dolphin-emu/source/detail?r=5a2d8661d7113ed2fd1c51d5f4e10b5ce385459a&name=new-ax-hle

Basically, when I'll be done with AX, AXWii, Zelda, ZeldaWii and JAC, HLE should be as good as LLE on 99.9% of games. At the moment there are still some bugs with AX though, so don't expect that work to be done before 2013.

[Duke Nukem]

Holy shit this is awesome news, I think I'm gonna wet my pants. 2013 is gonna be a sweet Dolphin year.