If you never see the guy again/deal with him again, is there any rational reason to punish him?

Punishing anti-social behavior is only worth the cost if you actually have to face the person again and deal with them.  Or someone you know and care about does.

And what makes it anti-social?