Dolphin, the GameCube and Wii emulator - Forums

I'm not sure if this posts belongs here, apologies if not.

I was wondering about the possibility of compressing generic roms with rvz. I realize that a lot of the work and savings maye come from GC/wii specific stuff. however even without that I think it would be beneficial to have for the software preservation and overall emulation community.

currently the "go to" so to speak for rom compression is .chd and as I'm sure many are aware of, while it offers great compression. it suffers from both encode and decode overhead due to limited support for compression algorithms the most common being lzma and zlib. this is an issue actually briefly talked about in the blog post covering the release of rvz. (Dolphin Progress Report: May and June 2020 ) and it doesn't seem like support for zstd or other low overhead compression algorithms will be coming to chd for quite some time.

this presents a potential benefit to software preservation and individuals with a fair amount of roms, low end hardware users such as SBCs can often get hit pretty hard by the decode penalty. but it also would be benefical to those storing large quantities of roms as it allows rapid compression which can be a large factory when dealing with hundreds of roms.  

although people who may benefit from this using preservation is a small group, I believe it would still greatly benefit the first group of people if it would be possible to compress roms with rvz. even if no emulator supports it, assuming the rom is small enough, thanks to zstd compression it would be fairly trivial to temporarily uncompressed it, use it, then delete the uncompressed copy when done.

Thanks for your time.

I've made a quick patch that lets you do this with the `dolphin-tool` command-line program. Please try it out: https://github.com/dolphin-emu/dolphin/pull/10507

(03-12-2022, 01:34 PM)Quackdoc Wrote: [ -> ]even if no emulator supports it, assuming the rom is small enough, thanks to zstd compression it would be fairly trivial to temporarily uncompressed it, use it, then delete the uncompressed copy when done.

If this is your use case, I would recommend using a format that compresses the file in one go instead of dividing it into blocks like RVZ does. That way you will get better compression, and you get to use a simpler and hopefully more widely supported format than RVZ (which has a lot of GC/Wii-specific features).

I should also mention that RVZ has a limitation where the compressed size can't be larger than 16 GiB.

(03-12-2022, 07:26 PM)JosJuice Wrote: [ -> ]I've made a quick path that lets you do this with the `dolphin-tool` command-line program. Please try it out: https://github.com/dolphin-emu/dolphin/pull/10507


If this is your use case, I would recommend using a format that compresses the file in one go instead of dividing it into blocks like RVZ does. That way you will get better compression, and you get to use a simpler and hopefully more widely supported format than RVZ (which has a lot of GC/Wii-specific features).

I should also mention that RVZ has a limitation where the compressed size can't be larger than 16 GiB.

Thanks a lot! I've only tested on as single PS2 and a single PS3 game for now, since that's all I have dumped atm. however it compares quite nicely in compression ratio to chd, while both decode and encode time's are far smaller than chd.

This does make me wish to see more adoption of RVZ for sure. it works very well.

For perspective on my ryzen 2600 encoding and decoding of Atelier Ayesha a 16gb PS3 game. I benched it twice, once at 3400mhz for standard boost clock, and one at 1500mhz to try and simulate a lower end PC, (still better than many SBCs). of course in the testing I made sure that the reverted images had matching md5sums. so it's compressing and uncompressing accurately. and both compressed the 16gb game to 13gb.
a nice size difference for sure.


3400mhz
rvz encode 3m31s
rvz revert 4m26s
chd encode 8m52
chd revert 12m39

1500mhz
rvz encode 4m02s
rvz revert 4m29s
chd encode 19m33s
chd revert did not finish

Much thanks, and hopes to continued success.

By the way, it would be cool if there were a way to only compile dolphin-tool (and if it was availiable as a separate package in ppa, right now it's not even included)
It's a real pain to install all the required library for dolphin and recompile the whole thing on a headless linux server when you just want to turn some iso to rvz.
At least, with chd, you don't have to install mame-tools package and that's it.

(03-22-2022, 07:01 AM)Nixx Wrote: [ -> ]By the way, it would be cool if there were a way to only compile dolphin-tool

That should be possible by turning off the CMake options ENABLE_NOGUI and ENABLE_QT.

Unfortunately the dolphin-tool binary will still draw in quite large portions of Dolphin (though not the Qt GUI). Hopefully we'll be able to improve that at some point.

(03-22-2022, 07:01 AM)Nixx Wrote: [ -> ](and if it was availiable as a separate package in ppa, right now it's not even included)

You'll have to contact the PPA maintainers about that.

(03-22-2022, 07:01 AM)Nixx Wrote: [ -> ]By the way, it would be cool if there were a way to only compile dolphin-tool (and if it was availiable as a separate package in ppa, right now it's not even included)
It's a real pain to install all the required library for dolphin and recompile the whole thing on a headless linux server when you just want to turn some iso to rvz.
At least, with chd, you don't have to install mame-tools package and that's it.

I'll second that.  I've tried dolphin-tool in Windows and it works great so now I'd like to use it on my RHEL 8 servers, but I'd rather not install a boatload of dev libraries on a prod server.  A dolphin-tool binary for RHEL 8 (x64) or a simple way to build it would be most useful.

Build a podman container to hold the build environment perhaps?