The texture cache setting can safely be set to Fast. Any games needing something more accurate will automatically override the setting (unless you open the graphics settings while playing, so don't do that).
If you keep it to Middle, it won't do anything for games that don't need something that accurate. However, games that still need a more accurate setting will ignore that setting and force a more accurate setting anyway. So you're really just reducing your performance for no good reason.
If you keep it to Middle, it won't do anything for games that don't need something that accurate. However, games that still need a more accurate setting will ignore that setting and force a more accurate setting anyway. So you're really just reducing your performance for no good reason.