Oh dear, that will teach me to jump to conclusions without doing enough research. I only checked the post I merited.
You found that this person dont deserve the merit points you gave to him, because of this post other members gave him some points, but you can do nothing to cancel your act!!! This is why I believe the merit system should be reviewed.