No, what I get is social contract and backing by society.
He intentionally said YOU and not SOCIETY, because if it's ME setting the standards, that's easy to argue against. If it's SOCIETY setting the standards, that's much more difficult to argue against.
If I hire hitmen to do my dirty work, am I not responsible for the death of my neighbor? I suppose technically speaking, that'd be false, but still ugly (I didn't "pull the trigger" personally). If you use the voting mechanism to achieve legislation, you have a difficult time determining who the "hitmen" were. It's all well hidden and abstracted within the mechanism of statism.
I'm not disagreeing with you, just adding. If you want to get that technical, even if you pull the trigger, it's the bullet that killed him. That's why we talk about proximate causes vs. ultimate causes. The proximate cause was the bullet or the hitman, the ultimate cause would still be you in either case.