I've been using Dolphin 3.5-367 to play all my games full speed with no problem at all ( monster hunter tri, zelda twilight princess, tales of symphonia dawn of the new world, among other games), and I decided to try the newest release (3.5-1655) and my fps lowers a lot even with the same settings, for example in monster hunter tri my fps with the 367 build is locked at 30, while in the 1655 build it lowers to 22-25.
The right man in the wrong place can make all the difference in the world.