Just before the end of the year, we read Ian Ayres’s musings on prediction markets over at Freakonomics. Writing on his personal blog, Consensus Point Chief Scientist Robin Hanson responded to the post and elaborated on whether prediction markets better served as methods or forums:

1. How to pick city policies, vs. how to pick the mayor.
2. How to cook a meal, vs. how to pick a restaurant.
3. How to win a game, vs. how to decide which team won.
4. How to do a study, vs. how to pick a study to publish.

These are four examples of methods vs. forums. Methods are ways to do things; forums are ways to pick who decides what to do. Yes, in a sense forums are methods, since choosing who decides indirectly picks what to do. But that is what makes forums powerful; good forums induce people to find good methods. Good elections induces good city policies, good restaurant competition induces good cooking, good game rules induce good play, and good journal review induces good articles.

To me, prediction markets are mostly interesting as forums, not methods. Alas many seem to confuse the two.

Robin elegantly puts the history of the concept into context and dismisses the idea that the wisdom of the crowds serves as an equalizer; rather true wisdom is revealed by self-selecting experts with incentives. He then goes on to suggest that academic journals might not be the best forum for choosing forecasting methods.

“Prediction markets” started from speculative markets, e.g. stocks, where accuracy comes much less from non-expert participation and much more from participants with incentives to self-select as experts. Any team that considers itself expert enough can pay to prove itself, but in fact most teams stay away and prices tend to be dominated by real experts, who get paid and really know better than most.

Prediction markets aren’t about emphasizing ordinary Joes over credentialed bigshots; they are about emphasizing whomever tends to be right. Simple opinion averages maybe be reasonable indicators of crowd wisdom, but they have too little of the forum-ness of letting self-selected expert teams come to dominate.

It seems to me that when academics like Aryes call for academic studies of prediction markets as methods, instead of as forums, they are implicitly suggesting that current academic institutions should be the forum in we choose forecasting methods. If academic journals prefer a method, they suggest, that’s the method the world should use.

In contrast, I suggest prediction markets may be a better forum than academic journals for choosing forecasting methods. Maybe the world shouldn’t use a method just because academics say its great; maybe those impressed with a method should have to put their money where their mouth is and trade on that method’s forecasts in prediction markets. Maybe the rest of us should just accept prediction market prices as our best estimates; if and when prediction market prices become dominated by traders using a method, that is when the rest of us will have implicitly accepted that method as best.

How might the academy respond? Our guess is with skepticism. Care to bet?

Okay, maybe the title is a bit overblown, but Consensus Point Co-founder and Chief Technology Officer Ken Kittlitz was second author (with Johan Almenberg and Thomas Pfeiffer) on a recent study at Harvard’s Program for Evolutionary Dynamics involving the application of prediction markets to scientific publication:

Prediction markets are powerful forecasting tools. They have the potential to aggregate private information, to generate and disseminate a consensus among the market participants, and to provide incentives for information acquisition. These market functionalities can be very valuable for scientific research. Here, we report an experiment that examines the compatibility of prediction markets with the current practice of scientific publication. We investigated three settings. In the first setting, different pieces of information were disclosed to the public during the experiment. In the second setting, participants received private information. In the third setting, each piece of information was private at first, but was subsequently disclosed to the public. An automated, subsidizing market maker provided additional incentives for trading and mitigated liquidity problems. We find that the third setting combines the advantages of the first and second settings. Market performance was as good as in the setting with public information, and better than in the setting with private information. In contrast to the first setting, participants could benefit from information advantages. Thus the publication of information does not detract from the functionality of prediction markets. We conclude that for integrating prediction markets into the practice of scientific research it is of advantage to use subsidizing market makers, and to keep markets aligned with current publication practice.

Imagine our surprise that the experiment further validates the use of prediction markets as powerful forecasting tools.

Thesis on the effectiveness of Prediction Markets as a forecasting tool 

by Kai Trepte and Rajaram Narayanaswamy
Graduate students in the Engineering Systems Division at the Massachusetts Institute of Technology

Graduate students in the Engineering Systems Division at the Massachusetts Institute of Technology studied 20 Prediction Markets at General Mills, powered by Consensus Point, in order to determine the accuracy of Prediction Markets as a forecasting tool in a corporate setting.  According to the study, “Our findings clearly show that Prediction Markets are capable of developing very accurate forecasts, effectively aggregate information from multiple participants and may be able to provide improvement for long-range forecasting.”

Click here to read the full thesis.

Paper courtesy of Harvard’s Program for Evolutionary Dynamics

by Johan Almenberg, Ken Kittlitz and Thomas Pfeiffer

Abstract
Prediction markets are powerful forecasting tools. They have the potential to aggregate private information, to generate and disseminate a consensus among the market participants, and to provide incentives for information acquisition. These market functionalities can be very valuable for scientific research. Here, we report an experiment that examines the compatibility of prediction markets with the current practice of scientific publication. We investigated three settings. In the first setting, different pieces of information were disclosed to the public during the experiment. In the second setting, participants received private information. In the third setting, each piece of information was private at first, but was subsequently disclosed to the public. An automated, subsidizing market maker provided additional incentives for trading and mitigated liquidity problems. We find that the third setting combines the advantages of the first and second settings. Market performance was as good as in the setting with public information, and better than in the setting with private information. In contrast to the first setting, participants could benefit from information advantages. Thus the publication of information does not detract from the functionality of prediction markets. We conclude that for integrating prediction markets into the practice of scientific research it is of advantage to use subsidizing market makers, and to keep markets aligned with current publication practice.

Click here to download the full version of the paper.

Abstract: Building on the success of prediction markets at forecasting political elections and other matters of public interest, firms have made increasing use of prediction markets to help make business decisions. This Article explores the implications of prediction markets for corporate governance. Prediction markets can increase the flow of information, encourage truth telling by internal and external firm monitors, and create incentives for agents to act in the interest of their principals. The markets can thus serve as potentially efficient alternatives to other approaches to providing information, such as the Sarbanes-Oxley Act’s internal controls provisions. Prediction markets can also produce an avenue for insiders to profit on and thus reveal inside information while maintaining a level playing field in the market for a firm’s securities. This creates a harmless way around existing insider trading laws, undercutting the argument for the repeal of these laws. In addition, prediction markets can reduce agency costs by providing direct assessments of corporate policies, thus serving as an alternative or complement to shareholder voting as a means of disciplining corporate boards and managers. Prediction markets may thus be particularly useful for issues where agency costs are greatest, such as executive compensation. Deployment of these markets, whether voluntarily or perhaps someday as a result of legal mandates, could improve alignment between shareholders and managers on these issues better than other proposed reforms. These markets might also displace the business judgment rule because they can furnish contemporaneous and relatively objective benchmarks for courts to evaluate business decisions.

Click here to download the full version of the paper.

Abstract: Developing obtainable, clear and measurable work expectations early in the project planning process is an important part of successful project management. Converting these expectations into project milestones and communicating openly about progress toward them is crucial to every project’s success. Optimistic estimation biases of IT workers, poor estimating techniques and group politics can hinder communication and decrease the chances of success. A prediction market is a tool that might help project managers overcome these obstacles.

Prediction markets are online marketplaces that adapt many of the same structures found in stock markets to aggregate information about the probability of future events. These markets have produced reliable estimates in a variety of settings, including corporate environments. This presentation will describe the design, implementation and evaluation of a prediction market to support the communication needs of an IT project manager overseeing the implementation of a software system in a distributed team environment.

Click here to download the full version of the paper.

Abstract: In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We lever- age a unique data source of almost 2000 people’s subjective probability judgments on 2003 US National Football League games and compare with the “market probabilities” given by two different information markets on exactly the same events. We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

Click here to download the full version of the paper.