Last week NPR’s Morning Edition had a piece on President Obama’s poll numbers after the second debate. As their discussion turned to his chances of winning based on Intrade betting numbers,
MONTAGNE: OK, so got all those numbers. Finally, we checked the betting crowd. Intrade, which bills itself as the world’s relieving prediction market, runs an online betting service whose participants put the odds of Barack Obama winning the election at 65 percent over Mitt Romney at 35 percent.
GREENE: But, Renee, odds only mean so much. That same service said there was a 75 percent chance the U.S. Supreme Court would strike down a national healthcare law. And the court beat those odds.
Notice the bolded text and you can see the fallacy in Greene’s argument or his lack of understanding of probabilities.
First these are very uncertain events to predict and the outcome can depend on many different variables. Modeling methods like Monte-Carlo simulation and prediction markets like Intrade try to estimate the level of uncertainty (by placing a lower and upper bound on them). The net result is a probability distribution of different outcomes and in this case whether or not President Obama will win this November.
When a model states one outcome is more likely than the other and in reality the other outcome happens one should not treat this as failure of the model. If there is no doubt who will win, there is no uncertainty, then one does not need any of these tools. By definition there is uncertainty in the outcome. The modeling indicates one outcome (say, Obama win) is more likely than the other.
The model is doing its job perfectly well. What it really states is, “if we were to imagine million different ways of running the 2012 election, more of them show Obama win over Romney win”.
Another point is confusing results of previous model on an unrelated event with the current one. There is no relation between Supreme Court’s decision and election results.
To say, “Odds only mean so much”, as a way to dismiss all predictive models is just plain wrong.