Imagine a different scenario. There was no HOA when I bought the house and no mention of any future plans. I bought the house without any conditions. Now, one day there is a knock on my door and someone says "You're now a member of the new HOA. If you don't like it, move."
Tell me, do you fantasize all the time that you are that farmer? Do you fantasize that you were here before the US government? Or wait - maybe I have it wrong! Let me guess - that home you own, the one with the HOA - you've owned it for several centuries.
You see, you aren't that farmer. When you bought your house, not only did you enter into an agreement with the HOA, you decided to continue to be a US citizen and resident of the US in addition to being a resident of the HOA. And you decided to continue to pay your taxes, instead of moving out of the country.