When you play the me and you game, it's easy to shoot down the regulation system because it looks more arbitrary.
You don't get extra rights just because you bring some friends to the party.
Life is not a party. If people band together for their own protection, you need more than talk about rights to remove the protections they have created.