Ok so I checked the Wiki and somebody reported that you can tackle this issue by ticking Skip EFB Access from CPU. I can confirm that the slow down doesn't happen when this is ticked but this makes all the pointer based features unusable (cannot collect star bits, cannot select planets...)
So I guess it's not really an ideal solution...
So I guess it's not really an ideal solution...
