If you never see the guy again/deal with him again, is there any rational reason to punish him?
There is no guarantee that you would never see him again. Either way it's all about feelings (as estevo tried to explain to you). The minor good feeling I could obtain by receiving a small amount of money would not outweight the bad feeling of letting a greedy person get away with it. If you don't get that bad feeling that's fair enough, but the reason that most people do is that they have the genes that have been the winners in the game of evolution.