OK, the difference of empires through history is related to the culture and demographic of that population, try the same model of an empire with the same philosophy but on different demographic and you will simply fail... the problem of libertarians is that you compare the whole population/humanity to one ideal, that if pushed certain ideas upon in a certain way it will work, but at the end of the day you ignore the fact that  people differ, from the way they think, to their traditions, to their culture, to their color, to their.... this is why states and borders exist, and some kind of enforced regulation has to exist so people can live with each other.