You can repair any pitchmark on the green at any time.
You can only repair any divot off the green when it is no longer in your line of play, you cannot improve your lie - unless the divot was made after you ball stopped. For example, you hit a ball down the fairway, when you get there you find someone from an adjacent hole has hit his ball onto your fairway just in front of your ball. You let him play first so that he can get out of the way but he leaves a big divot. You can repair it because you are entitled to have your original lie.