I see that you're running a 4770K. I would recommend turning on the integrated graphics in your motherboard's BIOS options so that you can use Quick Sync instead of CPU or ShadowPlay for recording/streaming. Quick Sync has unbelievably good performance and should let you record at near-full speed, even at ridiculous resolutions.
I also have a patch here that will allow you to render at an internal resolution of 4K (3840x3168). This will give you unbeatable image quality. Your GTX 770 should be able to keep up in most games.
If you would like, I could compile the latest build on master for you with that patch applied so that you don't have to go through the trouble yourself.
Also, I'm still hoping to put together a patch that will allow for custom internal resolutions. It's not too difficult, but I've been busy with other projects!
I also have a patch here that will allow you to render at an internal resolution of 4K (3840x3168). This will give you unbeatable image quality. Your GTX 770 should be able to keep up in most games.
If you would like, I could compile the latest build on master for you with that patch applied so that you don't have to go through the trouble yourself.
Also, I'm still hoping to put together a patch that will allow for custom internal resolutions. It's not too difficult, but I've been busy with other projects!