There are a bunch of ways to acquire BTC without going through any official exchanges.  This forum, and BitCoin-OTC are some examples of many.

I think that the main reason centralized exchanges are popular is that they are, for the unexperienced user, loads safer than any other form of obtaining BTC.  Sure you could set up a decentralized exchange, and adding an Escrow feature to it would make it more secure than most of the systems already in place.  However you would not be able to eliminate scamers entirely.

Though the main reason is that there has not been a need for anything like this yet.

Didn't really answer your question, but at least this gave your thread a bump.