Dawn Keller, formerly with Best Buy, evaluates ROI of prediction markets and how business leaders need to consider the “cost of not doing” and “cost of alternatives” in this equation.

 Excerpt from Dawn Keller’s blog, The Answer is in the Crowd

From what I’ve seen in the marketplace (both first and second hand), Prediction Markets frequently face the same bottom line scrutiny as any other enterprise application, tool, or resource. How much value will it generate and when? Arguably, the business case should be extra tight when evaluating something new and unconventional. And prediction markets fit the bill. They are not yet widely adopted; they stem from newfangled trends such as crowd-sourcing; and most egregiously, they challenge traditional management orthodoxies.

 Here’s what companies often say:
This seems very promising, but I’m not sure it’s worth the cost.  What kind of returns can I expect?  What is the value of this new information?

Here’s the cost-benefit equation those comments imply:
Market value = market benefit - market cost
    or …
ROI = market benefit / market cost
   where …
        cost = price of Prediction Market solution + cost of internal time & resources
        benefit = value of the information (generated by the market)
Not to get hung up on the math, but these simple equations are missing at least three variables:

1. the cost of not doing
2. the cost of alternatives
3. the multiplying factor of the company’s management “skill”

The cost of not doing

Here, I’m simply flipping around one of the original questions.  Instead of only asking what is the cost of doing something, sophisticated leaders also evaluate the cost of not doing something.  In other words, what is the risk of passing on a particular opportunity, or ignoring a particular problem?  Those risks should be considered, and considered as costs.

Read more from Dawn’s blog on the ROI of prediction markets and how prediction markets can address your business needs.

The iPredict Prediction Market, powered by Consensus Point, proves its precision in forecasting events.  Here is what’s being said about the accuracy of iPredict.

As posted on October 9^th by Eric Crampton of Offsetting Behavior.

iPredict’s forecasting performance: awesome [updated]

Matt has been doing great work over at iPredict, New Zealand’s event stock market. His latest update: iPredict’s rather good forecasting accuracy since it’s opened. On binary contracts (events that either happen or don’t), folks far too often in the iPredict forums, or even the most bombastic prediction markets blogger, say that the market has failed if its price close to closing is far away from the final price. Not so. If a stock is trading at $0.90 for a $1.00 contract, if that kind of contract doesn’t close at zero 10% of the time, the market is seriously wrong. So we want prices to follow a 45 degree line. Here’s the picture.   

A bit of underpricing around the 30-60% range, but on the whole, things are looking very good. Matt goes through a bunch of other ways of judging performance. Go read the whole thing, then start trading! At least if you’re a Kiwi.

Update: Chris Masse is right: the Olympics markets are not the kinds of events for which we would expect good prediction market performance. I just have fun poking noses. I have the underlying data for iPredict from Matt that will let me see whether we’re closer to the 45 degree line for the kinds of contracts that are more like the Olympics one as compared to the kinds of contracts that are more likely, a priori, to aggregate wisdom of crowds. Will check as soon as I’m unburied from the mountain of grading.

White paper on The Ingenix Prediction Market by Ingenix

The Ingenix Prediction Market provides a platform for understanding the psychology that drives people’s actions.

Executive Summary

If your organization offers health care benefits, understanding members’ health conditions and hidden risks is fundamental to eliminating excess spending. But you also need to know who will respond to your messages, what actions they’ll take based on the options you provide, and the timing of those actions:

Clinical Data (A) + Predicted Actions (B) = Elimination of Excess Spending

Many organizations offer clinical data. Where do you obtain the knowledge needed to predict human behavior?

Organizational decision makers in every market need to understand the psychology of their members in order to predict their actions. In the changing landscape of health care—with historical data offering few clues to the future—predicting behaviors related to health care options is the crucial challenge.

Knowledge, attitudes, and beliefs are directly related to behaviors. Understanding these factors is an issue for health care providers, researchers, insurers, pharmaceutical companies, businesses that offer health care benefits, and the employees who use health care. Fortunately, we need look no further than this diverse set of individuals to overcome the obstacles and get a clear vision of the future.

Retail organizations have successfully harnessed prediction markets to quantify the actions their employees and stakeholders will take and improve revenue forecasting as much as 70 percent over traditional forecasting methods.1 Best Buy, General Electric, and Hewlett-Packard—to name a few corporations that use prediction markets—have discovered a way to understand the behavior of their employees and stakeholders. This allows the companies to not only reduce the risk of bad decisions, but to accelerate innovation.

Now, the Ingenix Prediction Market brings this unique and reliable forecasting method to the health care sector, at exactly the moment when predicting reactions and driving innovation are desperately needed.

A More Accurate Approach to Research

A prediction market improves on traditional research methods by:

-       Providing a quantitative method to capture the attitudes and beliefs that influence behavior

-       Adjusting for respondents’ confidence and historical accuracy

-       Incentivizing respondents to reveal what they will do

-       Focusing on what groups of respondents will do rather than what an individual thinks

Predicting an Uncertain Future

A prediction market, also called a “decision market,” poses questions to a group of stakeholders, who respond with opinions of what is most likely to happen in the future. The stronger the opinion, the greater the number of points stakeholders allocate to their position. This can be done anonymously to ensure a candid response. Within a company, this means executive decision makers have access to opinions from the entire workforce, who otherwise might be reluctant or unable to share what they know.

 “The overarching problem every company faces is uncertainty,” says Ron Hoffner, Ingenix associate, “and they try a variety of ways to decrease that uncertainty.” The traditional approach is market research, using historical data, surveys, focus groups, and polls. “Each one of these methods has problems,” says Hoffner, “and the problems are accelerated in the rapidly changing world of health care. Using historical data is especially risky because the fundamental assumption is that the past will predict the future. In light of changing legislation, for instance, we cannot make that assumption.”

Focus groups, surveys, and polls present challenges, particularly within organizations, where individuals are not rewarded for delivering bad news. Multiple layers separate the people who have good information from the decision makers. A prediction market improves communication flow—within an organization or across an entire market.

Unlike polling and surveying, prediction markets are designed to predict the actions a population will take and illuminate the attitudes driving this behavior. Questions are designed in such a way that participants’ answers predict behavior; and participants are incentivized by earning points for disclosing the true attitudes of the population toward the topic. According to Hoffner, “When you want a way to summarize multiple points of view into actionable metrics, the appropriate question can gather those views and predict the likeliest outcome.”

Rev Up Your Research Engine

If you’ve done a Google search, you’ve participated in a prediction market. Based on a search of three billion possibilities, the Google search engine quickly predicts what web pages will be most useful to you. Google continuously improves its accuracy by calculating which sites are chosen most often.

Internally, Google uses prediction markets to determine whether or not product launch dates will be hit, new office openings, and other strategically important events.2 The Iowa Electronics Market (IEM) is a prediction market that was founded in 1988 as a way of predicting the outcome of political elections as measured by how individuals would vote. The IEM has proven more reliable than major national polls, even months in advance of an election. Similar markets have been created for other fields. The film industry has the Hollywood Stock Exchange, to predict Oscar winners and box-office results.

Until the Ingenix Prediction Market was unveiled, there were relatively few applications in health care. Pharmaceutical companies use them to improve awareness of project performance and for marketing. The Iowa Health Prediction Market calls on health care workers to help predict the spread of infectious diseases, such as the H1N1 flu. And now, in collaboration with Consensus Point, a leading provider of enterprise prediction markets, Ingenix has created a prediction market for the health care sector.

According to Robin Hanson, chief scientist for Consensus Point, prediction markets eliminate information bias by tapping diverse minds. Even intelligent individuals, says Hanson, are subject to the groupthink of peers. “Because we often disagree with other groups, we band together and end up agreeing too much with our own teams. No single leader can overcome such biases and data gaps to predict with certainty whether an action will succeed or fail.” Prediction markets produce forecasted behaviors that incorporate more information, run continuously, and tap the minds of those who otherwise would not provide opinions.

Collective intelligence is more reliable than any individual expert opinion. Expert or not, an individual can be wrong. In a prediction market, however, when all the responses are collected, the correct answer is remarkably close to the average of the responses. This has proven true in case after case, giving prediction markets a strong advantage over other forecasting tools.

How can a group of people have the right answer even when many individuals in the group are incorrect? In his book The Wisdom of Crowds, James Surowiecki explains: “If you ask a large enough group of diverse, independent people to make a prediction or estimate a probability and then average those estimates, the errors each of them makes in coming up with an answer will cancel each other out. Each person’s guess, you might say, has two components: information and error. Subtract the error, and you’re left with information.”

The Power of the Ingenix Prediction Market

The Ingenix Prediction Market is the first comprehensive market focused on health care. Ingenix’s strategic partner, Consensus Point, has helped a diverse array of clients improve their ability to make market-based predictions. Best Buy, CNBC, General Electric, General Mills, Motorola, and the Department of Defense have benefited from Consensus Point prediction markets that are tailored to their needs, utilizing the vast intellectual capital of employees and other stakeholders. The Ingenix Prediction Market is available online, with members of the health care sector participating and paying close attention to this constantly updated source of information. Ingenix can further customize this model to enable an organization to make market-based predictions about the actions of its unique member population.

“In the Ingenix Prediction Market, we give participants points that they can assign to the different questions based on their views,” says Hoffner. “The questions are tied to health care events and programs that may or may not be implemented. The more confidence the participant has in his or her opinion, the more points can be applied, because an individual can answer the same question multiple times.However, it costs a certain number of points to answer each time, and you can answer until you run out of points. Thus, individuals who are more confident in their view have more influence. In turn, participants are rewarded for being right by earning more points. The resulting value is a real-time indicator of the attitudes of a population, the actions they will take, and what is driving those decisions.”

Employers Use Prediction Market to Take Swift Action

Q: How will employees respond to a new health care payer?
A large research organization is switching to a new health care payer. Management needs to predict enrollment in each of the three new plans and to gauge members’ satisfaction with the new payer. Historical data and surveys can’t predict enrollment in new plans. Traditional methods take too much time, restricting the organization’s ability to take corrective action to resolve member satisfaction issues.

A: By using a customized Ingenix Prediction Market, management is able to obtain an early and accurate forecast of benefit plan enrollment numbers, and get a real-time read of employee satisfaction with the new plan.

Q: What can be done now to improve employees’ health and cut costs?
An employer needs an in-depth analysis of employees’ health needs, but also to identify what actions employees would be willing to take immediately regarding health issues such as obesity, alcohol use, and smoking.

A: Ingenix pairs a traditional survey with a Prediction Market. In four months, the survey provides an in-depth analysis of employees’ attitudes and health needs. However, in just two weeks, the Prediction Market reveals what actions employees are willing to take today to improve their health—allowing a potentially life-saving three-month jump on better health and lower costs.

For Information: 800.765.6696 | insight@ingenix.com Ingenix, Inc. | 12125 Technology Dr. | Eden Prairie, MN 55344 

Conclusion

While no one can be certain what the future of health care will look like, the need for strategic planning and sound decision making remains. The valuable information provided by Ingenix Prediction Market allows for innovative and proactive planning by policymakers, researchers, health care providers, and any organization whose business operations are affected by the issue of health care.

 “Everyone benefits by participating and sharing information,” says Hoffner. In addition to weighing in with opinions and gathering accurate, real-time information, participants in the Ingenix Prediction Market can pose their own questions. To explore how the Prediction Market works, visit www.ingenixpm.com and become a participant. If you’re interested in creating a customized prediction market for your organization, call 800-765-6089. 

About the Company

Ingenix is a global health care information, technology and consulting leader. We serve a diverse customer base within the health care community, including payers, physicians and hospitals, employers, pharmaceutical companies, consumers, property and casualty insurers, and government agencies. The number of ways that we improve health care is growing every day. Many of the most impactful innovations in health care are taking shape at Ingenix. We are applying the power of information to make the future healthier for everyone. 

Resources
www.consensuspoint.com 
1 Consensus Point Blog, April 9, 2008. Accessed July 30, 2009. Available at: www.consensuspoint.com/prediction-markets-blog/betting-to-improve-the-odds. 

Kunz, Ben, “Prediction Markets Meet Wall Street,” BusinessWeek, Oct. 14, 2008. Available at www.businessweek.com/technology/content/oct2008/tc20081013_033687.htm.

Surowiecki, James, The Wisdom of Crowds, New York: Random House, 2005.

The information in this document is subject to change without notice. This documentation contains proprietary information, which is protected by U.S. and international copyright. All rights are reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, without the express written permission of Ingenix, Inc. Copyright 2009 Ingenix, Inc.

Ingenix, a Consensus Point partner, to feature the Ingenix Prediction Market at the upcoming NBGH National Forum on Health, Productivity, and Human Capital, October 13-15, 2009 in Philadelphia. 

A prediction market improves on traditional research methods by:

-          Providing a quantitative method to capture the attitudes and beliefs that influence behavior

-          Adjusting for respondents’ confidence and historical accuracy

-          Incentivizing respondents to reveal what they will do

-          Focusing on what groups of respondents will do rather than what an individual thinks 

If your organization offers health care benefits, understanding members’ health conditions and hidden risks is fundamental to eliminating excess spending.  But you also need to know who will respond to your messages, what actions they will take based on the options you provide, and the timing of those actions.

The Ingenix Prediction Market brings a unique and reliable forecasting method to the health care sector, at exactly the moment when predicting reactions and driving innovation are desperately needed.

Excerpt from Michael J. Mauboussin’s Think Twice: Harnessing the Power of Counterintuition

Accurately projecting holiday sales is a crucial task for retailers.  A forecast that is too low leaves shelves bare and profits lost, while too much optimism leads to dusty inventory and pressure on profit margins.  So retailers have come up with a precise sales estimate.  To do so, most merchants rely on experts—individuals in the organization who gather information, study trends, and make predictions. 

                The stakes are especially high for consumer electronics firms because they generate so much of their revenue during the gift-giving season and the value of their inventory depreciates rapidly.  The pressure is really on the internal experts at consumer-electronics giant Best Buy, one of a multitude of retailers that rely on specialists.  So you can imagine the reaction when James Surowiecki, author of the best-selling book The Wisdom of Crowds strolled into Best Buy’s headquarters and delivered a startling message: a relatively uninformed crowd could predict better than the firm’s best seers.

                Surowiecki’s message resonated with Jeff Severt’s, an executive then running Best Buy’s gift-card business.  Severts wondered whether the idea would really work in a corporate setting, so he gave a few hundred people in the organization some basic background information and asked them to forecast February 2005 gift-card sales.  When he tallied the results in March, the average of the nearly 200 respondents was 99.5 percent accurate.  His team’s official forecast was off by five percentage points.  The crowd was better, but was it a fluke?

                Later that year, Severts set up a central location for employees to submit and update their estimates of sales from Thanksgiving through year-end.  More than three hundred employees participated and Severts kept track of the crowd’s collective guess.  When the dust settled in early 2006, he revealed that the official forecast of the internal experts was 93 percent accurate, while the presumed amateur crowd was off by only one-tenth of 1 percent. 

                Best Buy subsequently allocated additional resources to its prediction market, called TagTrade.  The market has yielded useful insights for managers through the more than two thousand employees who have made tens of thousands of trades on topics ranging from customer satisfaction scores to store openings to movie sales.  For instance, in Early 2008, TagTrade indicated that sales of a new service package for laptops would be disappointing when compared with the formal forecast.  When early results confirmed the prediction, the company pulled the offering and relaunched it in the fall.  While far from flawless, the prediction market has been more accurate than the experts a majority of the time and has provided management with information it would not have had otherwise.