![\"pit\"](\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2016\/06\/pit.jpg\")
<\/a><\/p>\nTwo recent events in the UK made it look like prediction markets’ predictions aren’t worth much.<\/p>\n
The soccer team Leicester City won the premiere league title despite the markets putting the odds of them doing so at 5,000 to 1 (.02%).<\/p>\n
Last week, people in the UK, voted to leave the European Union. A few hours before it was sure they would exit, a prediction market put their probability of leaving at 10%. See the figure above from PredictIt<\/a>. X axis is roughly time before the outcome was certain. Y axis can be interpreted as probability of exit (70 cents = 70%).\u00a0 It jumped from 10% to 90% in just five hours.<\/p>\n Analysts like to “explain” market results, coming up with a reason why an event was a failure of the prediction market. For instance, in the two events above, the Wall Street Journal, perhaps correctly, claims the bets were unduly influenced by London bettors<\/a>. Through big London bets the odds moved to reflect what Londoners believe instead of the sentiment of the crowd. In predicting a Brexit, the sentiment of the crowd is exactly what you want.<\/p>\n Whenever the prediction market is far on the wrong side of 50%, explanations will arise as to why the prediction market was wrong. Let’s take a step back here.<\/p>\n A desirable property of a prediction market is that it is calibrated. To be calibrated, events that it predicts to be 90% likely should occur 90% of the time. Events that it predicts to be 10% likely should occur 10% of the time.<\/p>\n If events that it predicts to be 10% likely (e.g. Brexit) occur 0% of the time, the prediction market has a problem. It is over-estimating the chances.<\/p>\n Looking at the calibration of prediction markets across many events, one will see that they are typically very well calibrated. Take, for instance that Hypermind<\/a> prediction market. The figure below shows close to 500 events that it predicted. If the market were perfectly calibrated, the points would fall along the diagonal line (i.e., 10% likely events would happen 10% of the time, 20% likely events would happen 20% of the time, and so on).<\/p>\n It’s very well calibrated. Here’s a similar chart<\/a> for U.S. primaries at PredictWise.<\/p>\n So the next time a market “misses” a 10% prediction, let us keep in mind that it needs to miss 10% of those predictions to stay calibrated. As the \u00c9mile Servan-Schreiber mentions in this post<\/a>, “It is perhaps a bit ironic to note that the data from the Brexit question slightly improved [the prediction market’s] overall calibration. It is as if the occurence of an unlikely event was long overdue in order to better match predicted probabilities to observed outcomes!”<\/p>\n![\"pit2\"](\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2016\/06\/pit2.png\")
<\/a><\/p>\n