It doesn't matter what I want, the laws of physics require that if I'm living with someone else, then we are sharing a set of rules together. I'm sure you have rules for other people, such as a rule by which if they do violence against you, you'll defend your self. This is ruling other people. So, in that way, you seek to rule other people that live in your shared space.

Anarchist, sovcit, and related philosophies that go beyond traditional libertarian have the mind boggling problem of circling into a quasi-dictatorship pushed into by freedom maximalism. Their personal rules for others must be followed by those others around them, and other people may not set any rules for them. Yet, that other person would claim the same against them, resulting in a clear and substantial societal conflict. This is typically an issue with people who set freedom as their highest virtue. Whether this would be a more localized problem would depend on how wealthy the people are who are involved in the dispute.

Put another way, there is a clear logical contradiction in individual self-rule. Two people may not agree one the finer lines of peace and violence, which naturally involves other people as both individuals seek to assert what they believe to be their rights. The resolution in my philosophy is to not hold the anarchist or sovcit flavor of libertarianism as my system of values.