That begs the question, if people are unable to trust each other, how can they trust a third party? (Especially if that third party is someone they've never met, never talked to, never voted for, and enforces arbitrary rules on transactions while taking an exorbitant fee for their "services" which in many cases are neither requested or desired.)
The fact that they have no stake in the transaction is marginally helpful, but my main point is that in the case of escrow defecting, you have two parties telling the same story as opposed to one person's word to another's in a direct trade.