It's vital that you stay 'behind the ball' when using the driver.

If you really do set up with the ball in the middle of the stance, sounds to me as if your weight must stay on the right side throughout the swing. Now, you can certainly hit straight shots like this, but you'll never generate the power you're really capable of.