We’ve been reading Harvard Business Review blogger and MIT Center for Digital Business researcher Andrew McAfee’s excellent book, Enterprise 2.0, which is full of valuable lessons for the enterprise, including that prediction markets are a very useful collaborative tool.

For instance, here’s an interesting discovery from the Google Prediction Markets, originally proposed internally in December 2004:

Analyses … revealed that at every point in time, even as much as ten weeks away from the closing date of the market, the most expensive outcome was the one most likely to actually occur. It seemed that GPM’s markets, in other words, could quickly and accurately distinguish among possible outcomes, identify the one most likely to occur, and attach a high price to that outcome.

This is exactly what our Foresight platform does on a regular basis for our customers.

Regular readers might remember a few months back when we cross-posted one of his posts from the Harvard Business Review blog. You might also recall when we posted a presentation that Linda Rebrovick (our CEO) gave in Chicago at the Prediction Markets Cluster conference in Chicago last November.

Linda noted the following best practice examples in her presentation:

• integrate into enterprise processes
• nurture executive sponsorship
• go big or go home
• make accessible to all
• customize to your business
• make it part of your value proposition

We were struck how similar these examples were to the Six Organizational Strategies identified by McAfee:

• Determine Desired Results
• Prepare for the Long Haul
• Communicate, Educate, and Evangelize
• Move into the Flow
• Measure Progress, not ROI
• Show That Enterprise 2.0 Is Valued

Coming back to the commentary on GPM, McAfee continues:

Google’s prediction markets shared with all markets a fundamental property: the ability to generate highly valuable information by bringing people together who have little or nothing in common.

Okay, we don’t actually know how different Linda and Andrew are, but we’re pleased that our executive leadership understood key lessons before an interested commentator went to press with his book. It’s almost… predictive.

On November 5 in Chicago, Dr. Robin Hanson, Chief Scientist of Consensus Point, and Rami Levy of Motorola, will be discussing the most effective applications of prediction and idea markets in business and government organizations at a lunch hosted by Consensus Point. Idea and prediction markets will become part of the internal DNA of the best organizations, as these solutions link human capital to organization results by providing leading indicators for the most important initiatives. Rami Levy of Motorola will share the business objectives and specific results of Motorola’s Thinktank Idea Exchange.  Dr. Hanson will share some specific approaches to structuring effective markets.

Robin Hanson, PhD., Associate Professor of Economics, George Mason University and Chief Scientist, Consensus Point

Rami Levy, Technical Lead and Manager, Open Source Technologies team, and distinguished member of Motorola’s technical staff, Motorola, Inc.

For more information, contact Consensus Point at info@consensuspoint.com.

Consensus Point and Our Customers, such as Best Buy and Ingenix, follow a proven enterprise methodology to yield maximum return and value from our prediction markets.  The major organizational best practices include:

1)   Define Business Problem that prediction markets solve and articulate value that could be derived with market insights; develop projected outcomes and track progress that prediction markets solve and articulate value that could be derived with market insights; develop projected outcomes and track progress  

2)    Determine Executive Sponsor and Prediction Market Leader – two different roles, Executive Sponsor guides market strategy and decision acceleration and Prediction Market Leader provides day to day leadership; incorporate prediction market reviews into existing steering committee structures

3)    Begin with a subset of participants, internal first, and validate accuracy through comparisons to baselines; run a market for minimum 90 days, then gather proof points and expand further to other employees, partners, and potentially customers

4)    Align incentives with company culture- and provide multiple rewards that include influence and career development incentives

5)    Determine mix of participants from several functional groups, with a minimum of 30-50 participants; the broader the participant base, the more liquidity for the market

6)   Be Transparent: communicate early and often about market success and decisions that are made from market insights; employee engagement will likely increase from hearing the actions being taken as a result of the market predictions

Prediction Market Reveals Answer Well Ahead of Results

April 28, 2009 – Ingenix.com - Ingenix Thought Leadership – When trying to determine whether a product is priced right, a project will launch on schedule or the impact proposed health reforms will have, health care entities are considering a methodology that relies less on expert opinions and more on the intelligence of hundreds of people with diverse vantage points.

To determine the results of a key question – what will the employee engagement index be? — on an employee survey, Ingenix used the Prediction Market to gauge the outcome well in advance of the survey results being published. Within one day, the Prediction Market accurately forecasted the index.  Prediction markets provides Ingenix’s clients with early and accurate quantitative data points on future events.

This methodology, known as “collective intelligence,” uses a “prediction market” to “unleash the collective wisdom of organizations and helps them gain a more accurate picture of what is going on in their organization and the industry,” according to Ron Hoffner, associate, Ingenix Consulting.

Prediction markets, defined in a recent BusinessWeek article as “bets on ideas,” ask groups of stakeholders the following question: “What do you think something is worth, and more important, what will it be worth tomorrow?”¹ The groups’ answers have proven to be accurate most of the time, the article goes on to say. “When groups of people bet on something, their combined intelligence is often remarkably prescient.”²

Hoffner agrees. “There is an old Japanese proverb that states ‘None of us is as smart as all of us,’ and we are finding more and more that this is true,” he said. Recent academic research has shown that using a market to forecast demand results in a significant improvement in forecast accuracy when compared to traditional methods.³ The result, Ingenix believes, is that health care programs can better predict which programs are the most likely to succeed.

Another area where prediction markets can add significant value is providing quantitative data on the impact proposed health reforms will have on the industry. “What we are doing is asking a variety of people with different perspectives – who traditionally have not been included in the discussion – for their input,” Hoffner said, “For really the first time, we are going to have quantitative data on the impact of these proposed health reforms that participating organizations can use for planning.”  

Prediction markets harness viewpoints, intelligence

One of the flaws of traditional forecasting is that it generally relies on historical data to make predictions about upcoming events and results. Many years ago, a wise man once said, “You can never plan the future by the past.”⁴ Indeed, Hoffner asserted, using historical data to determine the future assumes that the environment stays the same. However, the health care environment is rapidly changing and traditional methods that use historical data do not account for these changes. The result is forecasts with greater uncertainty.

“To illustrate this point,” Hoffner said, “let’s assume historical data shows two variables are related, such as gas prices and sports utility vehicle (SUV) sales. When gas prices go up, the sales of SUVs go down; conversely, when gas prices go down, SUV sales should pick up. However, they haven’t, so the forecast of SUV sales ended up being higher than the actual SUV sales results. It is clear that how variables are related in the future can change.” These changes can come from a variety of sources, such as social changes or government regulations, he explained.

Prediction markets also may be more reliable because they aggregate and consolidate data “from many individuals, often widely dispersed, each with access to small, idiosyncratic bits of relevant information.”⁵ In a prediction market, companies ask stakeholders from across the spectrum to voice their opinions about future events and milestones by buying “stock” or using points to register their “vote,” according to Hoffner.

“Let’s say a company wants to determine how many people will sign up for their four health plans and if a new wellness program will decrease absenteeism. This is their prediction market. They give stakeholders points to assign to the different questions in the market based on their own views,” he said. “If they have a lot of confidence in the issue, they will assign more points to their answer, so instead of just a  ‘yes’ or ‘no’  answer from a survey where everyone has equal influence, people with more confidence in their view have more influence by assigning more points in the market.” 

Another improvement prediction markets offer over traditional surveys is that participants are rewarded for being right. The result of answering a question right is the participant earning points. At the end of a quarter, for example, those with the highest number of points would receive an award. This results in participants seeking information on programs, which leads to increased awareness of them. 

Further, because the information derived from these diverse sources is contributed on an anonymous basis, the data collected also may be more truthful than if solicited in person. A lower-level employee with relevant exposure to an issue being carefully tracked by the company likely would not feel comfortable telling – or even be invited to tell — his or her boss’ boss about existing problems that might affect forecasts. However, that employee likely would honestly report an experience-based lack of confidence in meeting a future target date in the prediction markets setting.

“Prediction market data fills a lot of information holes, especially when there are many unknown variables, because the diversity of ‘players’ leads to different views and different perspectives that can help reduce uncertainty,” Hoffner suggested. “It lets management know how confident people are in a given forecast while still retaining control over the final decision.”

Expanding use, value of prediction markets

Ingenix believes that prediction markets can help health care entities bridge data gaps so they can “see more and do more,” Hoffner said, so it has forged an exclusive partnership with prediction market pioneer Consensus Point to offer prediction market services to Ingenix clients. These clients either pose questions to an Ingenix prediction market or consult with Ingenix experts to build or improve a prediction market of their own.

“Although collective intelligence is not new, today we are taking that intelligence to the next level, where more value can be derived from it,” Hoffner explained. Ingenix Prediction Markets will combine prospective data from employers, brokers, consultants, providers, payers and academics with longitudinal data from Ingenix’s repositories of patient claims and consumer surveys to provide industry-leading market forecasts of how trends, such as health care reform efforts, will impact the health care industry.”

Prediction markets give Ingenix “the ability to let our clients know what’s out there and what’s coming down the road – in a way that is more accurate than any other trends method – so they can better manage their business,” Hoffner concluded. “Our approach to prediction markets is that Ingenix is providing clients with real-time focus groups that we are calling a ‘living leadership forum.’ Any changes in the views of the group and the marketplace are quickly detected, which means businesses are rarely caught off guard.”

¹ Kunz, Ben, “Prediction Markets Meet Wall Street,” BusinessWeek (Oct. 14, 2008)
² Id.
³Consensus Point
⁴ Burke, Edmund (1729-1797).
⁵ Consensus Point, “What is a Prediction Market?” (Web site accessed April 1, 2009). ,” uses a “prediction market” to “unleash the collective wisdom of organizations and helps them gain a more accurate picture of what is going on in their organization and the industry,” according to Ron Hoffner, associate, Ingenix Consulting.

Employees use prediction markets to vote up product ideas and productivity improvements they think should be selected for development

April 27, 2009 — CIO.com — The project: Deploy a prediction market to aggregate new business ideas suggested by Motorola employees and assess their viability. A prediction market is a system for forecasting the outcome of projects or events based on how willing individuals are to buy “stock” in them. Users buy shares to vote items up. Each item is evaluated based on how much it is “worth”: the higher the value, the more popular the idea.

The business case: Motorola sought to allow any employee the opportunity to propose ideas for new products, upgrades to current products, productivity improvements or cycle-time reductions, says Rami Levy, distinguished member of Motorola’s technical staff and a member of its open-source technology team, which manages the prediction markets for the company. In 2003, Motorola had built a system to collect ideas, called ThinkTank. But when thousands of suggestions poured in, the teams that were supposed to weed through them were overwhelmed.

Using prediction market technology, Motorola could engage employees in the selection process by letting them vote for the ideas they thought had the best business potential. The most popular ideas could then be selected for further study and eventually be developed.

First steps: Levy and his team worked with a variety of director-level managers, including senior VPs, to secure buy-in for the project. “ThinkTank was already being sponsored by an SVP,” he says, but they needed support from others who would have to produce the business-case and user scenarios for new product ideas. That sponsorship was crucial to obtaining cross-organizational participation and funding for the tool.

Once Levy’s team secured management support, they integrated prediction software from Consensus Point (called ThinkTank Idea eXchange, or TIX, internally), with its existing ThinkTank application. In a six-month pilot during 2007, they experimented with market parameters, such as how long to keep ideas in play and how to finance participants’ purchases.

Today, employees submit ideas to ThinkTank, where anyone within Motorola can vote on them. Ideas that receive at least five votes are eligible for TIX, where each idea is initially valued at $10 per share. Anyone who wants to participate gets $100,000 to start with to buy the stock of the ideas they like best. As employees buy or sell shares, the value of the idea rises—or not. After 30 days, an idea review team determines which of the top-valued ideas to pursue. Winners are judged based on their stock performance, and participants who hold stock in winning ideas get a bonus.

It typically takes 18 months to develop a product at Motorola, so the first product ideas vetted through TIX are expected to come to market this year, Levy says.

What to watch out for: Market parameters that work in one circumstance won’t necessarily work in another, says Levy. You have to fine-tune your system to your environment. Motorola decided to limit an idea to a month in TIX to ensure new ideas were always entering the market. Meanwhile, even though employees find participating fun, you need to get them involved and keep them engaged. Levy’s team ran ads on the company’s intranet, conducted user satisfaction surveys and incorporated social media into the system. “Socialize the experience,” he recommends, “by integrating user comments, tagging, recommendations and links to other information.”

By Kristin Burnham © 2008 CXO Media Inc.

David Greenfield over at the ZDNet Team Think blog points out eight pitfalls to avoid when implementing internal prediction markets.  These comments come from a recent recent roundtable run by McKinsey & Company.  Jeff Severts, an EVP at Best Buy and Consensus Point client, participated on the panel.

1. Garbage-in-Garbage-Out – Prediction markets aren’t magic. By their very nature it depends on the quality and quantity of participation. Even smart individuals with in sufficient access to the right data will make poor predictions.

2. Lack of participation – Prediction market initiatives require rich involvement on the part of informed, diverse individuals. More important than lacking the number of people are not providing individuals with the access to available information needed for making informed decisions about their investments

3. Watch the optimism – Positive news and feelings tended to drive outcomes higher. Google found that new employees, for example, who were more optimistic about the company or when the stock performed well traders tended to result in people betting that good things would happen to Google, says Bo Cowgill, product manager at Google who has managed the company’s prediction markets for two and a half years.

4. Avoid extreme events. Google noticed that its traders undervalued extreme events, whether they were good or bad. When Google posed contracts with multiple outcomes, such as forecasts about the number of Gmail users, —the highest and the lowest outcome happened more often than the market expected.

5. The water cooler effect. Google found that that beliefs tend to cluster together. Individuals who sat and worked alongside one another, down to feet and meters from one another, tended to bet in similar ways. Geography was more important than work relationships, socializing outside of work, and language.

6. Watch the enemy - Best Buy found that employees tended to underestimate the competition or to think they knew more about them than they actually did. “Our prediction markets have not had a very respectable accuracy on anything related to our main competitor,” says Jeff Severts the vice president and general manager of Geek Squad, the services arm of Besty Buy, the US consumer electronics retailer

7. Air cover is key –Corporations need to be willing that an important initiative may fail. Too often, that’s not the case, which calls for senior executive sponsorship. “Air cover is key or you’ll find yourself trading on what kind of casserole we’re having in the cafeteria on Thursday,” says Severts.

8. Legal Trouble – Prediction markets deployment broaden the accessibility to sensitive information. How the Security and Exchange Commission (SEC)  will view that matter is anybody’s guess. “Take the employee who sees a prediction market price on her dashboard and realizes, with some degree of confidence, that a certain drug is going to be a success,” says Todd Henderson, an assistant professor at the University of Chicago Law School, “Is it illegal if she trades on this information in the real stock market? Is she an insider because she now has information that only a few top people had before? What kind of disclosure obligations does that put on a US public company? Gambling laws are another issue. Should prediction markets be viewed as an unregulated form of betting? These are enormous question marks for US public companies.”