Setup is a very big fundelmental, and you are right to pay close attention to it. For driver unlike the
irons you really don't want the hands ahead of the ball at address, primarily for the reason you have already discovered, it makes the shoulders open, and the low point of the swing for driver is behind the ball, so move the hands back to the left pant leg and bump the hip slightly towards the target, this will allow the shoulders to square and give you a small tilt in the upper body.
Here is a picture of chip Beck with some reference lines.