From a standpoint where you want to make things equal for people, what he says makes sense.

However, in the real world, people own stuff and can do whatever they want with it. If I wanted to and had enough money, I could buy an entire street in central London, and then let it stay vacant while homeless people roam the streets. And it would be perfectly legal, although not very moral..