Half of Intrade Users Are Dumber Than Nate Silver, But All of Them Are Smarter

That’s how I’d summarize the ongoing scholarly debate between the use of polls and the use of prediction markets to forecast elections.

Every article that talks about the 538 guy prefaces how he “came to fame” with his “unique approach” based on his time as a baseball numbers guy, that gave him a “near perfect” record on his first hand at election forecasting. Visualizing Data remarks:

Using computer modeling to generate huge numbers of simulated elections (Monte Carlo method?) to help predict the outcomes of the various stages of the election, he rapidly developed a reputation for incredible accuracy, eventually correctly forecasting the winners of 49 of the 50 states and every one of the 35 Senate races [in 2008].

I had a couple of immediate thoughts to this description.

(1) Running large numbers of simulations is not new.
(2) Intrade did better.

At least, its first time (2008 was Silver’s “first” for his method). Intrade users in aggregate accurately predicted the Presidential race in all 50 states and in DC, 34 of the 35 Senate races, and the House races.

Theoretically, there is a good reason for Intrade to be so accurate. Its users put their personal finances at stake in the outcomes of their estimates. There are immediate monetary incentives associated with paying attention. Intrade users are basically paid for having accurate electoral insights. As such, Intrade users are likely to have more insight into the elections than, say, someone whose livelihood doesn’t depend on observing information useful to understanding the outcome.

But I’m particularly irked by the statement that there’s anything revolutionary about Nate Silver’s methods. Especially considering some of his more notorious blunders, like the UK elections, the Maine gay marriage vote, and the long term state of the economy, Silver’s reputation is to me a rather remarkable phenomenon for someone who is fundamentally an unremarkable forecaster. Developing probabilistic models of potential future outcomes is an old and standard practice of statisticians.

The question is, can anyone forecast when there’s going to be some blowback against Silver-mania?