I think there is no one to blame but the real scammers. People are just vigilant because as we all know there are a lot of scammers out there.

I think you are just getting frustrated because you are not a scammers and people are accusing you as a scammer, do not worry in time you will build your brand and people will know that you are legit.