I could've sworn it was the merchants who built the first roads.

Imagine this: the state went away and nobody worked the roads.  They fell into disrepair; nobody drove on the roads anymore and it really started to hurt businesses; nobody would visit the shops and you couldn't get anything delivered.

Who has the greatest monetary incentive to get the roads in shape?  If Business A gets their roads built faster than competing Business B, they'll have mad profits; there's a rush to get the roads in shape to get a leg up over the competition.  Business C decides to skimp out on the quality and gets their road-builders A to speed the process up.  Road-builders A knows that they will lose business to Road-builders B if they don't keep their standards, but are eager to make a quick buck.  It is later found out that the road to Business C is already crumbling; Business C loses more business than if they'd just made a quality road and pay the price of it, while Businesses A and B flourish with their quality roads, and refuse to do business with Road-builders A due to their previous incompetence; Road-builders A eventually lose business to B, C, and D, and the labor is scooped up into C, as they're in the metro area and have a ton more roads to build; eager for more work, many laborers of Road-builders A accept the offer.  Business flourishes in infrastructure, as every brick & mortar business needs it; the people have their roads and the businesses have their commerce and all is well.

The market works fine, no state required.  Who builds the roads?--the people who gain from the roads being built, that being, all of us.