Its probably a sign to stop pm'ing him. I think thats whats annoying him, you keep asking him to remove bad feedback. Im not a fan of the feedback system, but it is what it is. I was going to suggest that you pm him to see if you can work something out, but I have a feeling that he will just leave you another negative feedback, so you are in a tough position.