Quote from: gwern on February 20, 2014, 02:02:09 AM

Quote from: edmundedgar on February 20, 2014, 01:44:21 AM

This is actually a problem with a lot of situations where you are punished for getting out of line with the consensus. The market tends to settle on a winner early, usually the one that's easiest to get started with, then has a hard time switching off it, even when everybody can see that there are better options. MS Windows, PHP, Mt Gox, QWERTY keyboards, cities built on geological faults, etc etc etc. But this is built with a much stronger incentive to stay with the consensus than any of those, and actually deliberately discourages a coordinated attempt to switch to something else, so people may be stuck in the city even as they watch it burn...

I'm not sure this is as serious a problem as you think it is. I have a fair bit of practical experience with prediction markets (see http://www.gwern.net/Prediction%20markets ), and in the one I am currently competing on, the Good Judgment Project (part of the IARPA competition), there have already been some issues with judging and data sources. They still work. I'd expect the problems to be self-correcting (although I'm still reading the papers so I could be misunderstanding something).

So, suppose John Doe releases an annual contract for annual American GDP growth > 3%. It's the only such contract because it seems so straightforward and clear. Everything is going fine until one year, the number comes in on a knife's edge: if the GDP figure is seasonally adjusted, it's 3.01% and the bulls win, and if it's raw, it's 2.9% and the bears win. The right figure is seasonally adjusted since the non-adjusted version will be predictably wrong when the truly final figures come in years later, *but* everyone is terrified of 'the crowd' not understanding this subtlety, taking the raw number that all the newspapers are trumpeting, and punishing them for disagreeing, so they all vote for 2.9% and the correct forecasters are screwed.

And next year the same thing happens and the prediction market is terrible and cats are living with dogs?

No, not really. Next year, John Doe, having learned from the fiasco, will issue his next version with the tweaked wording 'American *seasonally-adjusted* GDP growth is >3%'. The problem disappears. Alternately: John Doe doesn't learn & does nothing differently, and instead, Jack Frost, angrily remembering how he was burned last year, issues his own variant; now the crowds have a clear interpretation of both contracts: Doe's contract is raw, and Frost's is seasonally-adjusted. Savvy predictors flock to Frost's.

Since everyone can create contracts, good contracts drive out bad ones.

I don't doubt that the market can sort out the problem of ambiguous or badly-worded contracts (to within reasonable tolerances) but that's not the problem I'm talking about. The key innovation in this proposal is to use a p2p system of consensus witnesses to settle the contracts. It looks to me like the p2p system will collapse into de-facto centralization, at which point the incentive system, which is supposed to be designed to incentivize getting to true outcomes, will actively prevent the market from correcting.