I really like this comment. (Stolen from Pshemo's profile; hope he doesn't mind).
Since I've occasionally been the "victim" of revenge downvoting, I shall post this as a disclaimer. My thoughts and feelings on the matter are so much more eloquently uttered by NullPointerException, in a meta stackoverflow post he wrote:
If I downvote your post, I will tell you why I did it. I follow my downvotes, so if you correct it (or I am corrected), I will remove the downvote.
Consequently, I would expect the same of anyone downvoting me. Don't be that person...
Side Note: The word "victim" is quoted because it has not been an enormous problem for me (I'm sure others get it far worse), but enough at least to leave me rather baffled.