I wear a glove when it is a little wet out. Any other time I don't bother. If you've got good grips your hand will not slip anyway. Also, I never wear a glove when putting regardless of the weather.

When it comes down to the glove itself, I buy the smallest glove I can get on my hand. The tighter the better. I'd say I've got average size hands (wedding band is size 10) but both gloves in my bag right now are size small.