When you play the me and you game, it's easy to shoot down the regulation system because it looks more arbitrary.
You don't get extra rights just because you bring some friends to the party.
No, what I get is social contract and backing by society.
He intentionally said YOU and not SOCIETY, because if it's ME setting the standards, that's easy to argue against. If it's SOCIETY setting the standards, that's much more difficult to argue against.