Just curious, but have you considered how expediently you can access a magazine kept in the same holster the gun is kept in?
Especially if you carry on your strong side?
I mean, if your gun is in your strong hand, and you absolutely positively need a speed reload (for whatever reason) - what hand will you use to grab the magazine and finish the reload? If you use your strong hand - what will you do with the gun? Will you transfer the gun to your weak hand, then use your strong hand to grab the mag, ram it home, then re-transfer the gun back to your strong hand? Or are you versed enough to flip flop your grip with no discernible firing/sight alignment issues? Can you reach the mag easily and quickly enough with your weak hand to accomplish a reload? (Either around front or around back?)
Have you ever practiced speed reloads with dummy ammo with your current set up? I'm going to go out on limb and presume that you have not.
-J-