There's a bug in Dolphin where certain games drop to 30 fps in RealXFB when they shouldn't. Please report this.
http://code.google.com/p/dolphin-emu/issues/
Please follow the issue report guidelines. I've already confirmed the issue myself, but you risk the issue being closed if it isn't followed properly. Other games I know that are affected are Super Smash Bros. Brawl, Super Mario Galaxy 1/2, and many, many others.