@galop1n: Hey! I'm not sure how I feel about your "deferred EFB access" idea, it sounds pretty much like something which has bad chances of being integrated in the main Dolphin tree (it'd just be another just-sort-of-useful option in the graphics dialog). You're free to try it out anyways of course, just making sure there's no bad surprise if we indeed happen to turn down an eventual PR of yours.
Fyi, if your goal is to speed up EFB accesses in D3D11, have you looked into OpenGL's EFB access cache, yet? D3D would likely benefit a lot from such a cache, and moving the cache from OpenGL-specific code to VideoCommon has been on our TODO list for a while. There just has never been anyone with spare time and motivation to work on it though.
Fyi, if your goal is to speed up EFB accesses in D3D11, have you looked into OpenGL's EFB access cache, yet? D3D would likely benefit a lot from such a cache, and moving the cache from OpenGL-specific code to VideoCommon has been on our TODO list for a while. There just has never been anyone with spare time and motivation to work on it though.