I had the same problem before and I was also worried that changing the email would make people suspect that the account had changed hands. I signed a message to prove that I am the main owner, and in your case you can do the same as a persuasive step to convince people.

I don't think there will be much of a problem if you change the account email, but it is true that some people may be suspicious about the possibility of the account changing hands. You can also read some of the suggestions in my previous thread, it might help you.

Quick question about the account _BlackStar