Asking around for ideas about netplay

[Arisotura]

Hello there! I'm Arisotura, leader of the melonDS project. I have also worked on Dolphin in the past under another name (Luigi__ -- I was one of the people behind the original Zelda ucode implementation, for example).

Anyway, I'm coming here to ask around about one thing: netplay. Apologies in advance if this thread isn't in the right forum.


First, a bit of background. If you have followed the melonDS project, you might know that I was able to get DS local multiplayer working in melonDS back in 2017, but it was always finicky, unstable, and prone to data loss. For most of the time we put it on the backburner, because we knew it required tight sync, and we had other fronts to work on in the melonDS project, so local multiplayer just kind of sat there.

Fast forward to 2022. I have been working on it again, and managed to get local multiplayer connections stable and smooth at a decent level of performance.

So this gave us high hopes regarding DS multiplayer.


Naturally, the next step is to implement netplay. I don't know much about that, though, so I figured I would ask around emulators that already support it.


I hear about things like input prediction, rollback, ... Does Dolphin implement these things, and if so, how do you make it work reliably and fast with the emulation?

Does it deal with types of input more complicated than simple on/off buttons? I'm asking because with the DS, we got things like touchscreen input, microphone input, ...

Any other details regarding the netplay implementation that would be worth mentioning?

I figure I could just go read Dolphin's code, but it can also be good to hear from the devs ^^


Oh and there's another thing that's related to this: DS/Wii connectivity is sometimes brought up to us. It reportedly uses the same protocol as regular DS local multiplayer, but has any work been done on the Wii side of this, or is it uncharted territory?

[JosJuice]

Thanks for reaching out!

I hear about things like input prediction, rollback, ... Does Dolphin implement these things, and if so, how do you make it work reliably and fast with the emulation?

No, we don't have this. Rollback is a feature that's requested by users, but Dolphin's savestates are so slow that it isn't realistic to implement the way things look right now. It's a simple lockstep implementation, with a buffer that the user has to configure the size of.

Does it deal with types of input more complicated than simple on/off buttons? I'm asking because with the DS, we got things like touchscreen input, microphone input, ...

Yes. For GameCube, we deal with analog sticks and analog triggers, and for Wii Remotes we deal with an accelerometer, a gyroscope, an IR camera, extension controllers...

The handling we do for GameCube controllers is really very simple. Where you would send 1 bit of data for each button, you send 8 bits of data for each analog axis. Wii Remotes are much worse, but this isn't really because of all the exotic input methods, but because of details of how the Bluetooth communication between Wii Remotes and Wiis works. For instance, a Wii Remote can be configured to send data using different reporting modes, and our netplay implementation just completely breaks if there's a desync and two computers end up with different reporting modes for the Wii Remotes. Of course, desyncs mean you're kind of screwed anyway, but this limitation in our netplay code will prevent you from even saving your progress to continue later! But I digress. I'm assuming there isn't anything quite like reporting modes on the DS.

Oh and there's another thing that's related to this: DS/Wii connectivity is sometimes brought up to us. It reportedly uses the same protocol as regular DS local multiplayer, but has any work been done on the Wii side of this, or is it uncharted territory?

More or less uncharted territory. The only work that has been done on our side was to set up some stubs here: https://github.com/dolphin-emu/dolphin/pull/9300

[Arisotura]

Well, thanks for the reply!


Regarding complicated inputs, I was asking in the case you supported any sort of rollback, but since you don't, well...

"I'm assuming there isn't anything quite like reporting modes on the DS." -- indeed, there isn't. Well, there are a few games that make use of a special cartridge with a Bluetooth module, but we don't emulate that yet, and I don't think it's going to be relevant regarding multiplayer.

Other than that, the buttons are fairly simple, it's just a couple registers telling whether these are pressed. Touchscreen goes through a SPI controller, but nothing bad. Microphone might be trickier, seeing as on the DS there isn't any sort of hardware sampling or buffering for microphone input. The microphone is connected to one of the input lines on the touchscreen controller, and that just returns whatever the current microphone input level is. Then games have to do their own sampling to get data they can work with. Pretty barebones. But I digress.

The touchscreen bit made me think of a related thing: firmware settings, and also the BIOS files. Since we're going to have to run every melonDS instance locally on every player's computer, we do have to deal with these and ensure the setups are identical on all sides. Does Dolphin have to deal with anything of that sort, be it with simple Gamecube/Wii netplay or setups that involve GBA connectivity?


Regarding DS/Wii connectivity, I looked into the code and saw that there's a stub in place for these things. I think the challenge in this will come from having to interface Dolphin's HLE implementation to melonDS's wifi support, which is LLE. Where Dolphin deals with concepts like 'scan for a DS', melonDS deals with concepts like 'send the 802.11 frame at 0x46B0'. So basically Dolphin would have to reproduce all the low-level 802.11 stuff: sending beacons, authenticating and associating with clients, and then handling the local-multiplayer data exchanges.

I talked about this with JMC4789. From what I gather, during DS/Wii connectivity, the Wii acts as the host, which is good.

Basically, in Nintendo's local multiplayer wifi protocol, you get a host and up to 15 clients. Regularly (like, every 4ms or so), the host will initiate a data exchange, where it sends a CMD frame to all connected clients. Then the clients who need to reply automatically send a preconfigured reply frame (this part is assisted by the hardware, which automatically knows when to send the reply frame). Then the host sends an ack frame, which lets the clients know if other clients failed to reply.

So as the communication is driven by the host, the sync system I implemented in melonDS is also driven by the host. Frames that are sent are simply given microsecond timestamps, that clients can use to determine when to actually receive the frame and how long they can keep running for until they need to wait for more data from the host. This ensures that clients never run too far ahead of the host. Similarly, the host will discard any stale reply frames from clients, so that it doesn't run too far ahead of them.

Currently, the IPC between different melonDS instances is implemented using a shared memory buffer and named semaphores. The older wifi iteration uses UDP sockets, but I'm not sure about keeping this in -- the shared memory implementation allowed me to add extra checks and workarounds to avoid lag during (imperfect) multiplayer comm, that can't easily be replicated with sockets.

Just throwing some info out there if someone feels like looking into this. Or I might end up looking into it, but not before I've finished what I want to do with melonDS.

[JosJuice]

The touchscreen bit made me think of a related thing: firmware settings, and also the BIOS files. Since we're going to have to run every melonDS instance locally on every player's computer, we do have to deal with these and ensure the setups are identical on all sides. Does Dolphin have to deal with anything of that sort, be it with simple Gamecube/Wii netplay or setups that involve GBA connectivity?

We have an option to sync certain settings right before the netplay session starts (sending them from the host to each guest), both emulator-specific settings that affect sync and also the SYSCONF settings (roughly equal to your firmware settings). For legal reasons, we never send BIOS-like files. Hashing them is certainly possible, though I think right now we only have netplay hashing implemented for the game itself and not for any BIOS-like files.