I think that you do not need to see the club hit the ball, you just need to be looking at the ball when it gets hit.
You need to see the ball be there one second and gone the next.
Otherwise you have looked away too early.