Just before the end of the year, we read Ian Ayres’s musings on prediction markets over at Freakonomics. Writing on his personal blog, Consensus Point Chief Scientist Robin Hanson responded to the post and elaborated on whether prediction markets better served as methods or forums:
- How to pick city policies, vs. how to pick the mayor.
- How to cook a meal, vs. how to pick a restaurant.
- How to win a game, vs. how to decide which team won.
- How to do a study, vs. how to pick a study to publish.
These are four examples of methods vs. forums. Methods are ways to do things; forums are ways to pick who decides what to do. Yes, in a sense forums are methods, since choosing who decides indirectly picks what to do. But that is what makes forums powerful; good forums induce people to find good methods. Good elections induces good city policies, good restaurant competition induces good cooking, good game rules induce good play, and good journal review induces good articles.
To me, prediction markets are mostly interesting as forums, not methods. Alas many seem to confuse the two.
Robin elegantly puts the history of the concept into context and dismisses the idea that the wisdom of the crowds serves as an equalizer; rather true wisdom is revealed by self-selecting experts with incentives. He then goes on to suggest that academic journals might not be the best forum for choosing forecasting methods.
“Prediction markets” started from speculative markets, e.g. stocks, where accuracy comes much less from non-expert participation and much more from participants with incentives to self-select as experts. Any team that considers itself expert enough can pay to prove itself, but in fact most teams stay away and prices tend to be dominated by real experts, who get paid and really know better than most.
Prediction markets aren’t about emphasizing ordinary Joes over credentialed bigshots; they are about emphasizing whomever tends to be right. Simple opinion averages maybe be reasonable indicators of crowd wisdom, but they have too little of the forum-ness of letting self-selected expert teams come to dominate.
It seems to me that when academics like Aryes call for academic studies of prediction markets as methods, instead of as forums, they are implicitly suggesting that current academic institutions should be the forum in we choose forecasting methods. If academic journals prefer a method, they suggest, that’s the method the world should use.
In contrast, I suggest prediction markets may be a better forum than academic journals for choosing forecasting methods. Maybe the world shouldn’t use a method just because academics say its great; maybe those impressed with a method should have to put their money where their mouth is and trade on that method’s forecasts in prediction markets. Maybe the rest of us should just accept prediction market prices as our best estimates; if and when prediction market prices become dominated by traders using a method, that is when the rest of us will have implicitly accepted that method as best.
How might the academy respond? Our guess is with skepticism. Care to bet?