I know when I first started out the ball spent a while going every direction but straight. The guy teaching me focused on making sure that I always addressed the ball the same way. Making sure that I lined it up in the same spot that my shoulders were always square and that my grip remained exactly the same. Only then can you really start working with the face of the club he said. I spent a while fixing one thing, then breaking another. So don't give up.