We were delighted to see Roxana Strohmenger take up the notion of prediction markets as an opportunity for market research success. Citing Surowiecki, she gets straight to the heart of what makes prediction markets so powerful:

The beauty of this method is that it is not relying on asking individuals to make predictions about what they will do in the future but rather what they think other people will do. Research has shown that we are unreliable witnesses to our own motivations. However, as social animals, we are actually very good at noticing what other people are doing, sensing why they might be doing it, and predicting what they will do. Therefore, a crowd can successfully predict the future and ― for a market research professional’s purposes ― can predict which products will do well and which will fail in the marketplace.

She’s also clearly focused on the ROI impact for market research:

Prediction markets have typically been used during the discovery stage, when market researchers are ascertaining which ideas should continue on for extensive research and development. Rather than spending significant amounts of time and money on a target population using monadic testing for each idea put forward, researchers instead use prediction markets to quickly and inexpensively determine which idea is predicted to be the winning concept and then run that concept through the product development life cycle. When comparing against findings from monadic testing, companies that have used prediction markets have repeatedly found that the results are exactly the same in terms of what is deemed the winning concept. And instead of taking several weeks and paying significant amounts of money for these studies, they have reduced the research process down to several days and reduced their costs by the order of 50%. This is definitely a methodology that should become a staple of any market research professional’s tool kit.

This material was from her blog post, “A Cool Research Methodology That I Predict You Will Use,” from Jan 18, 2011. We look forward to keeping up with Ms. Strohmenger’s coverage of this sector.

If you’d like to see your brand serving as the focus of a case study on successful market research, get in touch.

Last week, we attended the Front End of Innovation conference in Boston. We went in part because the agenda looked interesting but also because we were pleased and proud to see two of our customers—Tina Brown-Stevenson of Ingenix and Rami Levy of Motorola—as presenters/panelists.

We enjoyed Tina’s presentation on prediction markets, and we’re glad that someone reported back to the FEI blog about her presentation:

In a solid FEI Champions presentation to complement James Surowiecki’s keynote address, practitioner Tina Brown-Stevenson, Sr. VP Innovation and Information Group at Ingenix, talked through the implementation of a prediction market at Ingenix. As a lead-in to the details of the specific practical application developed at Ingenix, Ms. Brown-Stevenson drew upon examples from Mr. Surowiecki’s book The Wisdom of Crowds to outline the foundational philosophical elements for Ingenix’s overall approach.

We won’t complain when a neutral third party favorably compares us to Surowiecki. Tina shared some findings from several prediction market uses and confirmed that the size of the crowd is not what is important— as markets can be accurate with as few as 24 participants. Tina confirmed what Surowiecki talks about in his book— it is the independence and diversity of the crowd that enables prediction markets to be on average more accurate than a subject matter expert. Tina also shared the Ingenix approach to adjusting the interface to their constituent needs. While many public companies, such as Best Buy and Motorola, have found the traditional stock metaphor to be adopted well within their cultures, we adjusted the Ingenix interface to something easily understandable for Ingenix participants, both internal and external. Demonstrates the flexibility of our Foresight platform!

For Rami’s presentation, he shared his notes with us from the panel on innovation adoption. He described Motorola’s idea collection system and explained why an idea market was needed: to identify the best ideas by leveraging the “crowd.”

For Motorola, the value of our Foresight platform boils down to three ‘Rs’ in the enterprise:

• rewards: mostly intrinsic, e.g., collaboration, helping the company, helping others, forming relationships, learning)
• recognition: via leader boards in the market and built-in social media tools
• recreation: fun, like a game; easy to use with low barriers to entry; competition; social media access to enable “viral” messaging.

If you’d like a sense of the kind of bottom line value these three Rs offered Motorola, CFO.com has covered that topic.

We’re glad we went to the conference, and we’re glad that two of our customers are on the front end of innovation through their use of Foresight.

Excerpt from Michael J. Mauboussin’s Think Twice: Harnessing the Power of Counterintuition

Accurately projecting holiday sales is a crucial task for retailers.  A forecast that is too low leaves shelves bare and profits lost, while too much optimism leads to dusty inventory and pressure on profit margins.  So retailers have come up with a precise sales estimate.  To do so, most merchants rely on experts—individuals in the organization who gather information, study trends, and make predictions. 

                The stakes are especially high for consumer electronics firms because they generate so much of their revenue during the gift-giving season and the value of their inventory depreciates rapidly.  The pressure is really on the internal experts at consumer-electronics giant Best Buy, one of a multitude of retailers that rely on specialists.  So you can imagine the reaction when James Surowiecki, author of the best-selling book The Wisdom of Crowds strolled into Best Buy’s headquarters and delivered a startling message: a relatively uninformed crowd could predict better than the firm’s best seers.

                Surowiecki’s message resonated with Jeff Severt’s, an executive then running Best Buy’s gift-card business.  Severts wondered whether the idea would really work in a corporate setting, so he gave a few hundred people in the organization some basic background information and asked them to forecast February 2005 gift-card sales.  When he tallied the results in March, the average of the nearly 200 respondents was 99.5 percent accurate.  His team’s official forecast was off by five percentage points.  The crowd was better, but was it a fluke?

                Later that year, Severts set up a central location for employees to submit and update their estimates of sales from Thanksgiving through year-end.  More than three hundred employees participated and Severts kept track of the crowd’s collective guess.  When the dust settled in early 2006, he revealed that the official forecast of the internal experts was 93 percent accurate, while the presumed amateur crowd was off by only one-tenth of 1 percent. 

                Best Buy subsequently allocated additional resources to its prediction market, called TagTrade.  The market has yielded useful insights for managers through the more than two thousand employees who have made tens of thousands of trades on topics ranging from customer satisfaction scores to store openings to movie sales.  For instance, in Early 2008, TagTrade indicated that sales of a new service package for laptops would be disappointing when compared with the formal forecast.  When early results confirmed the prediction, the company pulled the offering and relaunched it in the fall.  While far from flawless, the prediction market has been more accurate than the experts a majority of the time and has provided management with information it would not have had otherwise.

Ben Kunz just published a BusinessWeek article with several interesting insights about how the Dow Jones industrial average signaled the recent market turmoil - very much like how a prediction market aggregates intelligence about future events.

How did Wall Street know what would happen? It acted like a prediction market, a pool of intelligence that can foresee the future. Prediction markets are simply bets on ideas: What do you think something is worth, and more important, what will it be worth tomorrow? When groups of people bet on something, their combined intelligence is often remarkably prescient.

As you may know, this is something that James Surowiecki discusses extensively in his book, The Wisdom of Crowds.  Ben also talked with Robin Hanson about why the traditional prediction methodologies fail:

The trouble with humans, it seems, is that even when we’re smart, we have access to imperfect information and follow the groupthink of our peers. Because we often disagree with other groups, we band together and end up agreeing too much with our own teams. No single leader can overcome such biases and data gaps to predict with certainty whether an action will succeed or fail. But Hanson suggests markets can do just that.

(Ben Kunz is director of strategic planning at Mediassociates, a media planning and internet strategy firm. He is author of the advertising strategy blog.)