[Unofficial] New Dolphin 5.0 CPU benchmark - results automatically updated!

[Nintendo Maniac 64]

DISCLAIMER: The purpose of this thread is to use Dolphin and luabench as a general CPU benchmark that can used to determine a CPU architecture's relative capabilities in emulation-based workloads; it is not intended to benchmark Dolphin itself and should not be viewed as an example of absolute performance in Dolphin (which typically is several times slower).

----------------------------------------------------------------

CLICK HERE to submit your benchmark results

Click here to view the current results



Delroth, the creator of the previous Dolphin benchmark, has put together a newer benchmark that should be more comparable to real-world performance in Dolphin.  With the release of Dolphin 5.0, I figured that now was an appropriate time to put together a new Dolphin Benchmark thread - but this time, taking a page from Overclock.net's Cinebench 15 benchmark thread, the results will automatically update without any input from myself!


When specifically benchmarking the per-GHz and IPC performance between different CPU architectures, please try to run your CPU at 2.0 GHz, 3.0 GHz, or 4.0GHz. In order to make comparing easier, I have and will also mark all 2.0GHz, 3.0GHz, and 4.0GHz runs with inverted colors (white text with a dark background) on the spreadsheet. Thank you for your cooperation.


WARNING: Do not pause the benchmark at any time - doing this will actually hurt your final score!

Also note: Ignore the error [string ''IUB ambiguity codes''] while running the benchmark - this is NORMAL!

Screenshot of said error: (Show Spoiler)

----------------------------------------------------------------


[Windows only] "For dummies" edition - not for paranoid users!
(Paranoid users should instead scroll down and read the instructions under "Windows geeks that know what they're doing")

Download the following EXE (paranoid users should notice that it's merely a self-extracting 7z archive, meaning you can browse the file contents via any archive manager that supports 7z):

• https://drive.google.com/file/d/0B3hUE5vj6-14NHJ6RlREbVZ6eDA

Run the EXE and click "Extract".  Open the newly-created "Dolphin 5.0 CPU Benchmark" folder and double-click "Run Benchmark".

Select "Yes" or "No" if you want to send performance analytics to the Dolphin developers, then wait for the benchmark to complete.

Once "Overall time" is displayed, the benchmark is complete; take a screenshot and submit your results via the form above.  If this forum thread hasn't had any new posts in a while, then please consider making a post in this thread as well.





Other platforms + Windows geeks that know what they're doing

Download a copy of Dolphin 5.0-9 (which, according to JosJuice, is functionally identical to Dolphin 5.0 stable):

• https://dolphin-emu.org/download/dev/8a1bbaa56382b1bca3b95d79c716fc377c78e591

Once you've launched the actual program, go into "Config" and set the "Speed Limit" to 'Unlimited' (optionally you may also want to go into the "Audio" tab and set the "Audio Backend" to 'No audio output').

Also go into the "Graphics" settings, find the "Hacks" tab, enable the "External Frame Buffer (XFB)" and set it to 'Real' (Windows users may want to change their graphics "Backend" to 'Direct3D 11' which is necessary for many older GPUs and might also be a bit faster).

Now download the following Wii homebrew application and run it in Dolphin; this is the actual CPU benchmark:
http://delroth.net/luabench.dol ~ Mirror download link

Once "Overall time" is displayed, the benchmark is complete; take a screenshot and submit your results via the form above.  If this forum thread hasn't had any new posts in a while, then please consider making a post in this thread as well.

[mimimi]

When and where has delroth uploaded a new benchmark???

[Nintendo Maniac 64]

About 2 years ago, but for whatever reason he didn't seem to care about sharing the link publicly (you can find the link under the instructions for "Other Platforms".) I mean, the link worked for anybody, but he just never posted it anywhere.

For reference I found out about it when he PMed me the link about 2 years ago to see if I was interested in making a new Dolphin Benchmark thread, but I was waiting for the next major version of Dolphin before making the thread.

[mimimi]

Hmm, sounds like it was a work in progress, and that build is just a test version.

[Nintendo Maniac 64]

I don't think so...I'm pretty sure it was just a case of not wanting to manage the benchmark thread himself after throwing in the towel on the original official Dolphin Benchmark thread.

Besides, I had prematurely jumped the gun and had made a benchmark thread for the "failed" release candidates of Dolphin 5.0, and nothing was ever mentioned about the benchmark itself being unfinished or similar - just that Dolphin 5.0 itself was not actually ready at the time.


EDIT: Oh wow, I looked at my PMs wrong; he sent me the link 2 years ago; I've edited my previous post accordingly. If the development of luabench was "unfinished", then at this rate I don't think it'll ever be "finished".

[Helios]

(06-29-2016, 02:15 PM)Nintendo Maniac 64 Wrote: newer benchmark that is more comparable to real-world performance in Dolphin.

The benchmark is not representative of real world performance at all. By design. This benchmark is honestly useless for users beyond comparing JIT performance on different CPUs. Dolphin has many different workloads that can translate to different kinds of performance. GPU heavy games like SMG. MMU intensive games like the RS games. Games that require LLE. This benchmark doesn't really demonstrate any of that. All it tests is JIT performance. Thats it. And it doesn't even test it in a way that is useful to the user. As in, not gamespeed. Just time. A time number doesn't tell you anything about how good your computer is for dolphin beyond how much faster than a Wii it is (Which... Again, doesn't even tell you what speed you'll be playing at)

This benchmark is very useful to developers because it lets us see perf gains and regressions in the JIT, and *nothing else*. You'd also need to be testing on one hardware config for that to even be useful, which delroth did here: http://delroth.net/dolphin-benchmark.html

Combine that with the fact that you're telling users to go grab the latest build of Dolphin whenever they test and I question whether your benchmark result collections would even be useful (or readable) at all. There is zero consistency here. This isn't how you do a benchmark.

Changing the environment completely invalidates results.

[admin89]

Quote:The benchmark is not representative of real world performance at all

True . If I use Dx12 for demanding game like The Last Story , Quad Core Haswell i7 4700MQ @ 3.4GHz outperforms Dual Core Haswell G3258 @ 4.6GHz ...Simply because of multi-threading feature
It's the other way around in benchmark or when I use DX11/OpenGL though

[Kandira]

test done

[Nintendo Maniac 64]

(06-29-2016, 10:43 PM)Helios Wrote: The benchmark is not representative of real world performance at all.

I am merely quoting delroth himself; I hope he will not mind me quoting his PM:

delroth Wrote:

Nintendo Maniac 64 Wrote:

delroth Wrote:http://delroth.net/luabench.dol

First off, humor me, what makes this benchmark an improvement over the previous?

Should have an overall closer mapping to real world Dolphin performance, the previous bench was extremely floating point heavy even compared to games.

(06-29-2016, 10:43 PM)Helios Wrote: This benchmark is very useful to developers because it lets us see perf gains and regressions in the JIT, and *nothing else*. You'd also need to be testing on one hardware config for that to even be useful, which delroth did here: http://delroth.net/dolphin-benchmark.html

The purpose of this thread was to use Dolphin and luabench as a general CPU benchmark that could be used to determine a CPU architecture's relative capabilities in emulation-based workloads; it was never intended to benchmark Dolphin itself and, and you say, should not be viewed as an example of absolute performance in Dolphin as it is typically several times slower. This is also partially why I posted this thread in the "Hardware" sub-forum rather than the "General Discussion" sub-forum like the original benchmarks were.

If one remembers back to the launch of Haswell, all of the review sites found that it was only 5-10% faster than Ivy Bridge. However, nobody tested emulation workloads, so it wasn't until after-the-fact that forum members and the like discovered that Haswell was like 20-30% faster with emulation workloads - it was a performance jump large enough that other people initially thought it was a mistake and that something was wrong. It turns out that the results were very much legit and that, somehow, pretty much all of the CPU reviewers missed this.

(06-29-2016, 10:43 PM)Helios Wrote: Combine that with the fact that you're telling users to go grab the latest build of Dolphin whenever they test

May I ask where I say to get the latest build? All I see is where I say to "grab a fresh copy of Dolphin 5.0"...

[Admentus]

Ahh yes, "grab a fresh copy of Dolphin 5.0". I believe it is confusing because it may either refer to the stable or development builds. A fresh copy does not specifically which one it is. So which type is this benchmark aimed at? Stable or development? It is development... Well that's quite bad as it tends to be updated several times each day. The purpose for a benchmark is to have reliable results. Your intentions may be good, but the secret to sucess lies in phrasing it correctly.