Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)
JUST ABOUT EVERYONE GOT IT WRONG, SOME CLASSES OF PREDICTIONS WERE LESS WRONG
We know Decision Science News isn’t your main news source and assume you know that Donald Trump surprised many and won the election last night.
Models like the Princeton Election Consortium, which put Clinton’s probability of winning at 99%, probably need re-examining. Even PollyVote which averages polls, models, expert judgment, prediction markets, and citizen forecasts, forecast Clinton would win with 99% probability. It’s an average of 20 sources: none of which predicted Trump would win the most electoral votes. Historically, the average of many predictions is hard to beat.
The PredictIt prediction market (pictured above), mispredicted it though prediction markets weren’t that bad compared to other classes of forecast. In November, PredictIt was assigning Trump a 25-30% probability of winning. We bet against Trump on PredictIt when he was at 36% (2 or 3:1) and lost. This is sad for more than one reason.
Prediction market Hypermind (pictured below, lower graph is zoomed to November), fared similarly, giving Trump over a 25% chance in much of November (dates are written DD-MM-YY because Europe).
The Iowa Electronic Markets prediction market results are below. This is actually a winner take all market based on the popular vote plurality winner, but it’s close enough for jazz, meaning that people probably treat it the same as if it predicted the electoral vote winner(*). Note that this chart is on a different time scale (and we don’t have time to do anything about that), but focus on the period since August to compare to PredictIt and the period since October to compare to Hypermind. They had some volatility in predictions, going from 40% Trump down to 10% and back up to 40% a week before the election, though the average November prediction is comparable to PredictIt and Hypermind.
The summary is that all the prediction markets were wrong, but they weren’t steadily predicting 10:1 against Trump either.
Prediction market predictions were less wrong, going by something like Brier Score. Prediction markets predicted something near 20% to 25% Trump and a 4:1 or 3:1 horse won the race. As the French say ça arrive.
We could talk about more unique predictors like fivethirtyeight.com (below) which were volatile but still over 25% in November, and Keys to the White House, which is a simple tallying model that actually and barely predicted that Trump would win. However, we feel it’s better to talk about classes of predictions (like expert judgments or prediction markets or models) than unique cases. Also fivethirtyeight.com made three different forecasts, so, how fair is that?
(*) One interesting thing is that the IEM market was correctly predicting that Hillary would capture the majority of the popular (as opposed to electoral) vote going into the election. On election day, it moved the wrong way (predicting Trump would win the popular vote). The day after the election it predicted a 95% chance that Hillary would win the popular vote.