+- Dolphin, the GameCube and Wii emulator - Forums (https://forums.dolphin-emu.org)
+-- Forum: Dolphin Emulator Discussion and Support (https://forums.dolphin-emu.org/Forum-dolphin-emulator-discussion-and-support)
+--- Forum: Support (https://forums.dolphin-emu.org/Forum-support)
+--- Thread: Performance issues running Dolphin under KVM Hypervisor. No, really, this should work (/Thread-performance-issues-running-dolphin-under-kvm-hypervisor-no-really-this-should-work)
Performance issues running Dolphin under KVM Hypervisor. No, really, this should work - iphillips77 - 02-23-2016
Or, I think it should. Here's a brief rundown.
I've been using unRaid as a fileserver for years and years now. Running on a Slackware distribution, newer versions have integrated the KVM Hypervisor, a system for running virtual machines directly on the server. As I had a bunch of hardware that needed upgrading, I was pretty excited about this.. The prospect of running my fileserver, my gaming rig, and an upgrade for my old Mac all on one machine tucked away in the basement was great.
And, for the most part, it is. I'm currently running an i7-5820k overclocked to 4.5ghz on a Gigabyte X99-UD3P motherboard. 32 gigs of DDR4 RAM, a stupidly fast Samsung NVME SSD for virtual machine images, and a GTX760 passed directly to my Windows virtual machine. Good enough for a 7.9 Windows experience index, Cinebench OpenGL framerates comfortably over 100fps, and so on.
Now, before I embarked on this endeavour, I was told to expect about 5% overhead when gaming on a VM. Upgrading from a 2500k to the Haswell-E, I knew I was getting far more than 5% extra IPC with the new processor. Dolphin benchmark finished in around 5:30 at stock speeds, haven't benched since I've thrown an extra ghz at it. I should be pretty good here, right?
Of course, since I'm writing here, I'm obviously not. I'm getting some baffling symptoms, and they're always the same, regardless of most configuration changes. My test game is SMG1, which my 2500k+GTX760 had no trouble with. The game's title screen begins with a quick dip down to 55fps, then back up to 60, where it stays through the player selection screen. Starting a game, fps dip down to 40 or so, then climb back up. If I walk all the way to the left, then turn and come back again, there's a spot where the camera shifts and a sun/galaxy/whatever becomes visible, complete with some light ray effects. Here, fps drop to 45 and stay there as long as I stand there.
Now, you're saying, "oh well your system just can't handle it". Yeah, well. As I'm sitting there at 45fps, Dolphin is using less than 40% of my CPU. The two individual cores that Dolphin is using are sitting at around 60%. GPU is at 45%. GPU bus load at 22%. I've got so much headroom left, why isn't Dolphin using it? Here's a screenshot of what I'm looking at. FPS bouncing between 45-48 as I snapped this.
[attachment=14422]
Dolphin settings are pretty much standard, except I have to use render to main window or the game is completely unplayable. Rendering at standard resolution, no upscaling, no vsync, no funny stuff. DX11 and OpenGL act the same way. DX12 in a Windows 10 VM is slightly better but still acts the same, slowdowns in the same spots, etc.
I've got people over at the KVM forum discussing this as well over here, and come up with ideas that have helped other people with similar issues, but nothing has helped me, so I thought I'd ask around here.
Specific things about my setup that I thought would cause issues but I'm close to ruling out:
- Some of the virtual IO drivers I need to use increase DPC latency a bit. However disabling them doesn't improve things.
- 5820k doesn't have quite enough PCIe lanes, so my GPUs run at 8x instead of 16x. Bus load at 22% would suggest this isn't the problem, though. Especially because running at 3x resolution and 1080p doesn't make things worse.
- CPU isn't completely exposed to the host OS. There's no instruction translation or emulation, but Windows doesn't see all the cores -- just the cores I expose to it. As a result, I can keep other VMs or the host itself from interfering with Windows at all. However, Windows doesn't know things like bus speed, CPU speed, etc. CPU-Z reports my bus as being 1000mhz, and based on processor load CPU speed will be reported as 1, 2, or 3ghz. Could this possibly be throwing Dolphin off?
My apologies for the lengthy post, but I want to be thorough here. I'd appreciate any help anyone could provide.. and if there's any other information at all about my setup that I'm forgetting, I'd be happy to provide it if needed.
Thanks in advance!
RE: Performance issues running Dolphin under KVM Hypervisor. No, really, this should work - ExtremeDude2 - 02-23-2016
So you're running an emulator in an emulator?
RE: Performance issues running Dolphin under KVM Hypervisor. No, really, this should work - Helios - 02-24-2016
Not really, Extremedude2.
hypervisors using VT-x and d don't really emulate, of course I'm just being pedantic.
Anyways, I booted Dolphin just fine on my KVM setup.
What you're missing is that Dolphin does heavy work on two threads. The CPU emulation thread, and the GPU emulation thread. Dolphin cannot just magically "Use all the threads you give the OS"
You're either CPU thread limited, or GPU thread limited.
CPU thread limited tends to be the most common.
RE: Performance issues running Dolphin under KVM Hypervisor. No, really, this should work - KHg8m3r - 02-24-2016
I don't really know about KVM, but I do know Windows and Dolphin. In any other system I would say that the GPU thread can't keep up, since Mario Galaxy is a GPU heavy game.
What sort of GPU driver do you have in this setup, the default Nvidia drivers? If so, go to the Nvidia Control Panel, and set a high performance profile for Dolphin. I can only think that because of KVM, it isn't telling the GPU to go to full load.
RE: Performance issues running Dolphin under KVM Hypervisor... - iphillips77 - 02-24-2016
Quote:What you're missing is that Dolphin does heavy work on two threads. The CPU emulation thread, and the GPU emulation thread. Dolphin cannot just magically "Use all the threads you give the OS"
Oh yeah, I get that, which is why I mentioned that the two individual cores that Dolphin was using were stuck at around 60%.
Might have figured it out -- looks as though it could be a CPU frequency scaling issue. Ironically, if I had a weaker CPU, and it had to work harder for Dolphin, it would kick up to a higher frequency and run smoother. But it wasn't doing that for me. Tinkering with CPU governors on the Linux side of things has helped incredibly. Setting the minimum allowed frequency to be the same as the maximum, essentially telling those particular cores to run at maximum frequency at all times, has got things running much smoother now. Maybe not a long-term solution, but it shows that "running an emulator in an emulator" is possible!
In case anyone stumbles across this thread having the same issues as I am.. My system (and maybe yours) defaults to the 'powersave' governor. Check this by typing "cpufreq-info" at the terminal. My available governors are "powersave" and "performance".. You may see "ondemand" and "conservative" in there if you've got an older CPU, those have been deprecated for mine apparently. Here's the output of "cpufreq-info -c0", which just shows the settings for core 0 (they're individually customizable, you can have some cores running performance and others running powersave)
Code:
cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009
Report errors and bugs to cpufreq@vger.kernel.org, please.
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: 0.97 ms.
hardware limits: 1.20 GHz - 4.30 GHz
available cpufreq governors: performance, powersave
current policy: frequency should be within 4.30 GHz and 4.30 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency is 4.30 GHz (asserted by call to hardware).Yours will be different. My current policy has been changed by me so that core 0 always runs at 4.3ghz while I test this out. I've also changed the governor to 'performance'. Your hardware limits will be different as well, I'm overclocking my CPU to 4.3ghz.
Let's say you're passing CPUs 0-3 through to the VM. Give this a try.
Code:
cpufreq-set -c0 -d 4300000 -g performance
cpufreq-set -c1 -d 4300000 -g performance
cpufreq-set -c2 -d 4300000 -g performance
cpufreq-set -c3 -d 4300000 -g performanceThat'll set all four cores to run at maximum speed.
To set it back, just
Code:
cpufreq-set -c0 -d 1200000 -g powersave
cpufreq-set -c1 -d 1200000 -g powersave
cpufreq-set -c2 -d 1200000 -g powersave
cpufreq-set -c3 -d 1200000 -g powersave...the '4300000' and '1200000' derived from 4.3ghz and 1.2ghz, as indicated in the hardware limits section reported by cpufreq-info.
Just got a Dolphin benchmark time of 4:52 and some nice smooth 60fps SMG at 3x resolution.