In general, what you said sounds like a good way to do all kinds of betting services. People not releasing their funds, or not releasing them in a timely fashion, would be just contained in the rules of the game. Some people will probably not release their money even if they get something in return, but that could just be subtracted from the winners amounts. And in many cases the service provider would be position-neutral, and the site could be run with zero capital. I wonder why this kind of thing doesn't seem to be standard by now.
Even without the use of a mediator this kind of system seems quite good to me. The risk the user of a site like this takes would be that he wouldn't get any of his winnings, but would get his own money back. In an extreme case the operator of the site could refuse to release anybodies money, but he wouldn't get it either. The whole site could also be shut down by the authorities, with the same result.
I guess you fundamentally misunderstand how betting works. If I put a dollar on the line and you put a dollar on the line and then "I get my dollar back" I've just been scammed of a dollar, the one that used to be yours and you lost.
The idea that you're "protecting" my dollar by allowing the other party to steal my other dollar if they wish to do so is ridiculous. I wish no such protection, I wish to have the guarantee that the other party can't control their dollar, so I can win it. The other party wishes the same guarantee about me.
Basically betting is exactly the opposite of this scheme you're proposing. The fact that BitBet is doing thousands in bet BTC a month whereas an implementation of your idea doesn't even exist in practice should be sufficient empirical evidence that you're wrong. Why isn't it?
I think the method would do something valuable, although it is not a completely straightforward transaction which is based purely on trust.
The multisig protection would at least defend against a catastrophic loss, for example the site getting hacked and everybody losing their money.
Here is another example that jumped to my mind. It is also a kind of a bet. Person A makes a promise to send B a package in the mail and B makes a promise to pay one bitcoin. The other person is in effect betting on receiving the package and the other on getting the money. Person B pays 1.5 bitcoins to a multisig address. After he receives the package, he creates a transaction to pay 1 bitcoin to A and 0.5 bitcoins to himself. Both have an incentive to sign the transaction because they both get more than zero bitcoins.
The situation where either one of them refuses to sign the final transaction is a kind of an outlier which doesn't happen if the participants are acting rationally.