Prediction Market Reveals Answer Well Ahead of Results

April 28, 2009 – Ingenix.com - Ingenix Thought Leadership – When trying to determine whether a product is priced right, a project will launch on schedule or the impact proposed health reforms will have, health care entities are considering a methodology that relies less on expert opinions and more on the intelligence of hundreds of people with diverse vantage points.

To determine the results of a key question – what will the employee engagement index be? — on an employee survey, Ingenix used the Prediction Market to gauge the outcome well in advance of the survey results being published. Within one day, the Prediction Market accurately forecasted the index.  Prediction markets provides Ingenix’s clients with early and accurate quantitative data points on future events.

This methodology, known as “collective intelligence,” uses a “prediction market” to “unleash the collective wisdom of organizations and helps them gain a more accurate picture of what is going on in their organization and the industry,” according to Ron Hoffner, associate, Ingenix Consulting.

Prediction markets, defined in a recent BusinessWeek article as “bets on ideas,” ask groups of stakeholders the following question: “What do you think something is worth, and more important, what will it be worth tomorrow?”¹ The groups’ answers have proven to be accurate most of the time, the article goes on to say. “When groups of people bet on something, their combined intelligence is often remarkably prescient.”²

Hoffner agrees. “There is an old Japanese proverb that states ‘None of us is as smart as all of us,’ and we are finding more and more that this is true,” he said. Recent academic research has shown that using a market to forecast demand results in a significant improvement in forecast accuracy when compared to traditional methods.³ The result, Ingenix believes, is that health care programs can better predict which programs are the most likely to succeed.

Another area where prediction markets can add significant value is providing quantitative data on the impact proposed health reforms will have on the industry. “What we are doing is asking a variety of people with different perspectives – who traditionally have not been included in the discussion – for their input,” Hoffner said, “For really the first time, we are going to have quantitative data on the impact of these proposed health reforms that participating organizations can use for planning.”  

Prediction markets harness viewpoints, intelligence

One of the flaws of traditional forecasting is that it generally relies on historical data to make predictions about upcoming events and results. Many years ago, a wise man once said, “You can never plan the future by the past.”⁴ Indeed, Hoffner asserted, using historical data to determine the future assumes that the environment stays the same. However, the health care environment is rapidly changing and traditional methods that use historical data do not account for these changes. The result is forecasts with greater uncertainty.

“To illustrate this point,” Hoffner said, “let’s assume historical data shows two variables are related, such as gas prices and sports utility vehicle (SUV) sales. When gas prices go up, the sales of SUVs go down; conversely, when gas prices go down, SUV sales should pick up. However, they haven’t, so the forecast of SUV sales ended up being higher than the actual SUV sales results. It is clear that how variables are related in the future can change.” These changes can come from a variety of sources, such as social changes or government regulations, he explained.

Prediction markets also may be more reliable because they aggregate and consolidate data “from many individuals, often widely dispersed, each with access to small, idiosyncratic bits of relevant information.”⁵ In a prediction market, companies ask stakeholders from across the spectrum to voice their opinions about future events and milestones by buying “stock” or using points to register their “vote,” according to Hoffner.

“Let’s say a company wants to determine how many people will sign up for their four health plans and if a new wellness program will decrease absenteeism. This is their prediction market. They give stakeholders points to assign to the different questions in the market based on their own views,” he said. “If they have a lot of confidence in the issue, they will assign more points to their answer, so instead of just a  ‘yes’ or ‘no’  answer from a survey where everyone has equal influence, people with more confidence in their view have more influence by assigning more points in the market.” 

Another improvement prediction markets offer over traditional surveys is that participants are rewarded for being right. The result of answering a question right is the participant earning points. At the end of a quarter, for example, those with the highest number of points would receive an award. This results in participants seeking information on programs, which leads to increased awareness of them. 

Further, because the information derived from these diverse sources is contributed on an anonymous basis, the data collected also may be more truthful than if solicited in person. A lower-level employee with relevant exposure to an issue being carefully tracked by the company likely would not feel comfortable telling – or even be invited to tell — his or her boss’ boss about existing problems that might affect forecasts. However, that employee likely would honestly report an experience-based lack of confidence in meeting a future target date in the prediction markets setting.

“Prediction market data fills a lot of information holes, especially when there are many unknown variables, because the diversity of ‘players’ leads to different views and different perspectives that can help reduce uncertainty,” Hoffner suggested. “It lets management know how confident people are in a given forecast while still retaining control over the final decision.”

Expanding use, value of prediction markets

Ingenix believes that prediction markets can help health care entities bridge data gaps so they can “see more and do more,” Hoffner said, so it has forged an exclusive partnership with prediction market pioneer Consensus Point to offer prediction market services to Ingenix clients. These clients either pose questions to an Ingenix prediction market or consult with Ingenix experts to build or improve a prediction market of their own.

“Although collective intelligence is not new, today we are taking that intelligence to the next level, where more value can be derived from it,” Hoffner explained. Ingenix Prediction Markets will combine prospective data from employers, brokers, consultants, providers, payers and academics with longitudinal data from Ingenix’s repositories of patient claims and consumer surveys to provide industry-leading market forecasts of how trends, such as health care reform efforts, will impact the health care industry.”

Prediction markets give Ingenix “the ability to let our clients know what’s out there and what’s coming down the road – in a way that is more accurate than any other trends method – so they can better manage their business,” Hoffner concluded. “Our approach to prediction markets is that Ingenix is providing clients with real-time focus groups that we are calling a ‘living leadership forum.’ Any changes in the views of the group and the marketplace are quickly detected, which means businesses are rarely caught off guard.”

¹ Kunz, Ben, “Prediction Markets Meet Wall Street,” BusinessWeek (Oct. 14, 2008)
² Id.
³Consensus Point
⁴ Burke, Edmund (1729-1797).
⁵ Consensus Point, “What is a Prediction Market?” (Web site accessed April 1, 2009). ,” uses a “prediction market” to “unleash the collective wisdom of organizations and helps them gain a more accurate picture of what is going on in their organization and the industry,” according to Ron Hoffner, associate, Ingenix Consulting.

 License Agreement Provides Access to Successful GE Imagination Market, Comprehensive Innovation Management Process 

Nashville, TN – Consensus Point LLC, a leading provider of prediction market software and services announced the addition of Rebecca Whitehead Munn to the leadership team and a license agreement with General Electric (“GE”).   Through the GE relationship, Consensus Point customers can have access to the proven GE Imagination Market process and benefit by identifying the most promising ideas for business growth.  GE’s Imagination Markets help answer tough business questions such as “what new technology ideas should we be investing in” and “what new products should we develop”. Market participants can submit their own ideas for entry into the market, and they can buy and sell shares of any idea in the market based on how well they believe the idea will contribute to the market’s objectives. At the end of the market, the business leader has a rank-ordered list of ideas.

GE has implemented the Consensus Point solution for idea management in over 10 GE businesses since 2006. The GE Imagination Market has uncovered innovative ideas based upon a variety of business-designated criteria. By licensing and integrating the GE “Imagination Market” technology process, Consensus Point provides a proven, comprehensive design and solution leveraging the prediction market expertise of Consensus Point and the business innovation leadership of GE.  In the recent GE-wide Imagination Market, over 1,400 employees from 170 business segments representing 42 countries suggested over 220 business model innovation ideas. GE Imagination Markets provide a fun way to engage people globally in new idea generation and result in an abundance of relevant ideas that can aid in setting future strategic directions.

Rebecca Munn joined the Consensus Point team as Senior Vice President of Sales. She brings over 22 years of proven experience in strategic planning, sales and marketing leadership, business unit operations, and services delivery. At Consensus Point, Ms. Munn is leading sales, sales operations, marketing, and customer relationships. She has held leadership positions with several global service and technology companies, most recently as the Vice President of Sales Operations of Healthways.  Ms. Munn also held several leadership roles at Cisco Systems, including the former Director of the Services Division, Connected Health Vertical, and principal in the Internet Business Solutions Group. She was the co-founder of The Arbora Group, a retail and wholesale operations start-up, in 2001. Ms. Munn is a former board member of the Nashville Technology Council. She earned an executive MBA from the University of Colorado and a BBA in marketing from the University of Texas at Austin.

“Our customers now have access to GE’s global experience in innovation management and proven design for the Imagination Market, commented Linda Rebrovick, CEO, Consensus Point.  “This fully integrated solution, including the Consensus Point Foresight software and GE Imagination Market process, allows organizations to rapidly launch and realize the significant benefits of an idea and innovation market. Some of our customers have cut the time to process ideas in half through using idea markets. In addition to the GE partnership, we are pleased to announce the continued growth of our company with the addition of Rebecca Munn to our leadership team.” 

Employees use prediction markets to vote up product ideas and productivity improvements they think should be selected for development

April 27, 2009 — CIO.com — The project: Deploy a prediction market to aggregate new business ideas suggested by Motorola employees and assess their viability. A prediction market is a system for forecasting the outcome of projects or events based on how willing individuals are to buy “stock” in them. Users buy shares to vote items up. Each item is evaluated based on how much it is “worth”: the higher the value, the more popular the idea.

The business case: Motorola sought to allow any employee the opportunity to propose ideas for new products, upgrades to current products, productivity improvements or cycle-time reductions, says Rami Levy, distinguished member of Motorola’s technical staff and a member of its open-source technology team, which manages the prediction markets for the company. In 2003, Motorola had built a system to collect ideas, called ThinkTank. But when thousands of suggestions poured in, the teams that were supposed to weed through them were overwhelmed.

Using prediction market technology, Motorola could engage employees in the selection process by letting them vote for the ideas they thought had the best business potential. The most popular ideas could then be selected for further study and eventually be developed.

First steps: Levy and his team worked with a variety of director-level managers, including senior VPs, to secure buy-in for the project. “ThinkTank was already being sponsored by an SVP,” he says, but they needed support from others who would have to produce the business-case and user scenarios for new product ideas. That sponsorship was crucial to obtaining cross-organizational participation and funding for the tool.

Once Levy’s team secured management support, they integrated prediction software from Consensus Point (called ThinkTank Idea eXchange, or TIX, internally), with its existing ThinkTank application. In a six-month pilot during 2007, they experimented with market parameters, such as how long to keep ideas in play and how to finance participants’ purchases.

Today, employees submit ideas to ThinkTank, where anyone within Motorola can vote on them. Ideas that receive at least five votes are eligible for TIX, where each idea is initially valued at $10 per share. Anyone who wants to participate gets $100,000 to start with to buy the stock of the ideas they like best. As employees buy or sell shares, the value of the idea rises—or not. After 30 days, an idea review team determines which of the top-valued ideas to pursue. Winners are judged based on their stock performance, and participants who hold stock in winning ideas get a bonus.

It typically takes 18 months to develop a product at Motorola, so the first product ideas vetted through TIX are expected to come to market this year, Levy says.

What to watch out for: Market parameters that work in one circumstance won’t necessarily work in another, says Levy. You have to fine-tune your system to your environment. Motorola decided to limit an idea to a month in TIX to ensure new ideas were always entering the market. Meanwhile, even though employees find participating fun, you need to get them involved and keep them engaged. Levy’s team ran ads on the company’s intranet, conducted user satisfaction surveys and incorporated social media into the system. “Socialize the experience,” he recommends, “by integrating user comments, tagging, recommendations and links to other information.”

By Kristin Burnham © 2008 CXO Media Inc.

Ben Kunz just published a BusinessWeek article with several interesting insights about how the Dow Jones industrial average signaled the recent market turmoil - very much like how a prediction market aggregates intelligence about future events.

How did Wall Street know what would happen? It acted like a prediction market, a pool of intelligence that can foresee the future. Prediction markets are simply bets on ideas: What do you think something is worth, and more important, what will it be worth tomorrow? When groups of people bet on something, their combined intelligence is often remarkably prescient.

As you may know, this is something that James Surowiecki discusses extensively in his book, The Wisdom of Crowds.  Ben also talked with Robin Hanson about why the traditional prediction methodologies fail:

The trouble with humans, it seems, is that even when we’re smart, we have access to imperfect information and follow the groupthink of our peers. Because we often disagree with other groups, we band together and end up agreeing too much with our own teams. No single leader can overcome such biases and data gaps to predict with certainty whether an action will succeed or fail. But Hanson suggests markets can do just that.

(Ben Kunz is director of strategic planning at Mediassociates, a media planning and internet strategy firm. He is author of the advertising strategy blog.)

When executives at electronics retailer Best Buy Co. want to know if a new product or idea is likely to succeed, they can seek the opinion of rank-and-file employees by turning to the company’s “prediction market.”

The market, called TagTrade, allows Best Buy’s workers to trade imaginary stocks based on answers to managers’ questions. The market’s judgment has often proved to be more accurate than the company’s official forecasts.

TagTrade is open to all of Best Buy’s 115,000 U.S. employees. The roughly 2,100 of them who choose to participate get $1 million in fake money to trade for a nine-month period. The top trader in the period wins a $200 gift certificate.

Jeff Severts, a 38-year-old executive who is currently chief of the company’s Geek Squad personal-computer maintenance services, spearheaded development of the market. Mr. Severts says TagTrade helps flag potential problems early.

In January, he asked both his management team and the market to predict sales of a new service package the company was offering for laptop computers. A week before the company began selling the product, the market’s guess was 33% lower than the team’s official sales forecast. It subsequently sank further.

When initial sales figures confirmed the market’s prediction, Mr. Severts ended the offer and began redesigning the service package. He listed a TagTrade stock to gauge the revised package’s chances of starting on time, in mid-September. The stock rose, and Mr. Severts says he took “great comfort from that.” On Sunday, Best Buy started offering the new package.

Best Buy isn’t the only company using prediction markets as a way to tap the knowledge of front-line employees. Web-search giant Google Inc. uses them to solicit forecasts on everything from how many users its Gmail service will attract to whether products will launch on time. Other companies that have experimented with them include General Electric Co., Intel Corp. and Microsoft Corp.

Best Buy’s chief executive, Bradbury Anderson, encourages experiments like TagTrade that seek to drive decision-making down the corporate ladder and information up toward the top.

Last year, two marketing employees, working on their own, created an internal electronic bulletin board called Blue Shirt Nation, a reference to the trademark garb of the retailer’s sales force. The bulletin board’s 24,000 users post questions for co-workers, comments on work policies and personal videos.

Tim Sheehan, a senior vice president, watched the bulletin board closely after Best Buy decided to reduce its employee discount last year. Mr. Sheehan, who had participated in the decision, emailed fellow executives, quoting some of the fiery reactions that were posted on the site; the company reinstated the larger discount within days.

Mr. Sheehan now scans Blue Shirt Nation daily. In November, he helped start another site that lets employees with business ideas seek help and funding from elsewhere in the company, bypassing the normal chain of command.

Mr. Anderson, the CEO, says narrowing the gap between management and workers helps to make his Richfield, Minn.-based company more nimble and responsive to customers, while boosting sales and profits. Monday, Best Buy agreed to buy music service Napster for $121 million in what it said was a bid to reach new business.

To encourage more open lines of communication, Mr. Anderson, who says he remembers how distant corporate headquarters felt when he was a Best Buy clerk in the 1970s, invites management gurus like James Surowiecki, author of “The Wisdom of Crowds,” to address his employees.

Inspired by Mr. Surowiecki’s thesis that a crowd, collectively, can make better decisions than individuals — even experts — Mr. Severts decided three years ago to test whether Best Buy’s workers could outperform its bosses.

At the time, Mr. Severts was a marketing executive looking for ways to improve the accuracy of Best Buy’s sales forecasts. He emailed hundreds of Best Buy employees asking them to guess how many gift cards the electronics retailer would sell in February 2005.

As a group, the employees’ guess was more accurate than Best Buy’s official projection. But, Mr. Severts recalls, the forecasting director for gift-card sales said her team felt humiliated.

As a result, Mr. Severts says he learned to keep his experiments from seeming like personal attacks on the competence of his fellow managers. He explains his work at employee gatherings, using PowerPoint presentations that compare Best Buy to a centrally planned economy like the Soviet Union and feature portraits of Joseph Stalin and Mao Zedong. Those leaders, like many managers in big companies, were prone to bad decisions because they were too far from the front lines, Mr. Severts says.

In 2006, armed with Mr. Anderson’s support, a $50,000 budget and some volunteers, Mr. Severts widened his experiment to include a prediction market.

To spark interest in trading, Mr. Severts listed stocks on political contests and DVD sales; one asked how often Mr. Anderson, the CEO, would use the expression “lifestyle groups” during a conference call.

Mr. Severts moved slowly, wary of ruffling more feathers or forcing his ideas on skeptical managers. Seeking more credibility for his results, he enlisted researchers from the University of Chicago to study how market guesses compared with official forecasts. He hopes to get the results later this year.

Other Best Buy managers have started floating stocks. Earlier this year, Rob Rausch, a senior manager two levels below Mr. Severts, created a stock to gauge the likely reaction to a price increase for PC maintenance services. Managers of Best Buy’s private-label business floated a stock to help predict sales of a new digital-TV converter box.

Despite Mr. Anderson’s encouragement, employees say most of Best Buy’s management innovation still emerges from isolated pockets of the company, and depends on the benevolence of bosses who permit work on pet projects.

Mr. Severts says he is glad Mr. Anderson doesn’t force reluctant managers to change their ways. “Nothing would squeeze the life out of TagTrade faster than putting it at the top of a priority list,” he says.

One of Mr. Severts’s latest stocks asked if Best Buy’s information-technology department would finish a planned systems upgrade by mid-September. The market correctly forecast that the department would miss its deadline, but Mr. Severts noticed a glitch: most of the tech work was done by employees of consulting firm Accenture Ltd., who didn’t have access to TagTrade. He proposed opening the market to them as well; Accenture managers say they are interested in participating.

Copyright 2008 Dow Jones & Company, Inc.

What if your boss, rather than dismissing you off hand when you suggested a different way of doing something, let you bet on the fact that your idea was better. And paid you if you were right.

That is the premise of prediction markets – a tool companies are increasingly using to make better decisions by allowing employees to trade in a mock stock market based on information they have about the business.

They work a little like a futures market. Say a company wants to release a new product but is unsure whether a launch date can be achieved. It creates a market for a prediction that the product will launch in a given month. Employees are then given a fixed amount of a mock currency they then use to trade contracts in that future.

Those who think it is likely the product will launch in that month will buy the future, and the price goes up. Those who don’t, sell, sending the price down.

At the end of the trading period, the value of the contracts that have been bought and sold are tallied, and those traders who have been most successful are given cash prizes or bonuses.

The company benefits since the final ‘price’ reflects a broadly based forecast of the likelihood that a given launch date can be achieved.

Several large American firms – including Google, GE, Hewlett-Packard, and Best Buy – have begun using prediction markets to help make assessments about all kinds of projections, including how customer numbers will grow, what demand will be for a product, and when it will launch.

Companies can also set up more complex trading systems which allow employees not only to trade in a given prediction but also to make their own estimates of when they think a product will launch, say, and trade contracts in that future.

Two recent reports – by McKinsey, the management consultancy, and Forrester, the analyst – have suggested that such markets are likely to grow in importance, particularly as the technology which facilitates them becomes cheaper and more widespread.

The advantages, exponents say, are many: the trading system gives employees an incentive to share information they have that may be valuable: although the currency is artificial, the cash rewards are real.

A market also aggregates knowledge more efficiently, as employees can – in effect – give feedback to their boss by trading more or less aggressively on information they have. The social and cultural issues which may have prevented an employee from sharing information are also, in theory, swept aside, because all trades are anonymous.

Companies that have used prediction markets say they provide more accurate information about aspects of their business than could be learnt by more traditional methods, such as polling employees or consulting outside experts.

In one instance, a malfunction in the furnace of a chip manufacturer led employees at the factory to update their positions on future yield more quickly than could employees elsewhere.

Google, the search giant, has been using such markets since 2005 to estimate – among other things – traffic volumes, and when its international offices will open.

About 1,500 employees will trade at any one time, and each has 10,000 ‘Goobles’ – a mock currency – to play with per quarter. The Goobles are issued weekly, “otherwise people would lock up their positions early and there’d be liquidity issues,” Bo Cowgill, an economic analyst at Google, said.

The markets run for anywhere between two weeks and three months. When they close, successful traders are given cash prizes and specially branded T-shirts which, Mr Cowgill said, end up being a greater incentive than the money.

At GE, the healthcare to aviation and media conglomerate, about 40 to 50 predictions – typically about the types of new technologies the company should invest in – are traded at any one time by up to 10,000 of its 330,000 employees.

“We use them as another point in the decision-making process, alongside asking experts and other business leaders,” said Christina LaComb, a computer scientist in the R&D lab at GE.

The window displaying the live trading system sits on the desktop of the employee’s PC. The employee sees the current asking price for any contract and its trading history, and there are simple buttons for buy and sell.

One problem is the inherent biases of such markets, which can make the information they provide less useful.

“The first is a long-shot bias – people tend to overesimate the likelihood of a long-shot paying off, so they tend to overpay to make that bet; conversely they tend to underpay for a sure thing,” said Jeff Severts, a VP of services at the US electronics giant Best Buy, which has used prediction markets for two years.

“The second bias is a home-team bias: employees have been overpaying for ’stocks’ that are based on outcomes that would be good for Best Buy, but we believe we can mitigate the biases through continued refinement of the market.”

The US Commodity Future Trading Commission is also examining prediction markets from a regulatory point of view, and companies which use them have steered away from running markets which predict financial results.

“If we ended up with an application that did an amazingly good job or predicting our results, there would be a concern that anyone who saw it would be an insider,” said Mr Cowgill.

Todd Henderson, an assistant professor at University of Chicago Law School who has written about prediction markets, said that, assuming prediction markets were approved by regulators, the case for using them was compelling, and that the “real puzzle” was why every firm didn’t.

“Companies are the members of society most comfortable with markets as processors of information, and yet when it comes to decision-making, they display these socialist ‘command and control’-style tactics to make them,” he said. “Twenty years from now, prediction markets will be ubiquitous.”

Companies pay anywhere from $25,000 to upwards of a $1 million to run a market, according to David Perry, president of Consensus Point, a business which implements web-based internal trading systems.

CORPORATIONS live and die by ideas, and many enterprises have used Web-based technologies, like blogs, wikis and social networks, to gather thoughts and hasten their way into new services, products and cost-saving steps.

Now executives say they are harnessing a new Web tool, called prediction markets, to transform the idea pipelines inside their companies. Companies like the InterContinental Hotels Group, General Electric and Hewlett-Packard are using prediction markets to try to improve forecasting, reduce risk and accelerate innovation by tapping into the collective wisdom of the work force.

Like blogs and wikis, prediction markets can spur communication and collaboration within a company. Yet they add rigorous measurement to business forecasts, like estimating the sales of a new product or the chances that a project will be finished on time.

Corporate prediction markets work like this: Employees, and potentially outsiders, make their wagers over the Internet using virtual currency, betting anonymously. They bet on what they think will actually happen, not what they hope will happen or what the boss wants. The payoff for the most accurate players is typically a modest prize, cash or an iPod.

The early results are encouraging. “The potential is that prediction markets may be the thing that enables a big company to act more like a small, nimble company again,” said Jeffrey Severts, a vice president who oversees prediction markets at Best Buy, the electronics retailer.

The store chain has experimented with prediction markets on everything from demand for digital set-top boxes to store-opening dates. For example, Mr. Severts said that in the fall of 2006, the prices in a prediction market on whether a new store in Shanghai would open on time — in December 2006 — dropped sharply from $80 a share into the $40 to $50 range. Players made yes-no bets, and the virtual dollar drop reflected increasing doubt that the store would open on time.

Indeed, Best Buy’s first store in China opened late, in January 2007, but the warning signs from the prediction market helped prevent further slippage.

Mr. Severts noted that prices in a current prediction market — betting whether new offerings from its Geek Squad service will be introduced on time in June — are in the $90 range, an encouraging sign.

Best Buy plans to move beyond pilot projects in prediction markets to involve more workers throughout the company, starting next month. “It helps on two fronts, the speed and accuracy of information, so that management can move faster to deal with problems or exploit opportunities,” Mr. Severts said.

For years, public prediction markets have been used for politics, like the Iowa Electronic Markets and Intrade, where buyers and sellers bet on which candidate will win a particular race. And there are prediction markets where people place bets on news events (Hubdub, among others), video game sales (simExchange) or movie box-office receipts (Hollywood Stock Exchange).

These markets have often been more accurate than professional pollsters or market researchers. The idea is that the collected knowledge of many people, each with a different perspective, will almost surely be more accurate than an individual or small group or even experts. The concept has been championed by academic economists and was popularized by James Surowiecki’s 2004 book “The Wisdom of Crowds.”

Robin D. Hanson, an economist at George Mason University, proposes a “futarchy,” a form of government enhanced by prediction markets. Voters would decide broad goals of national welfare, but betting in speculative markets would determine the policy steps to achieve those goals.

Few in the corporate world go that far. An important issue is whether prediction markets are mainly an innovative way to gather information from employees or a font of reliable answers. “It’s still an open question whether the wisdom of crowds is really wise,” said John Kao, a consultant and the author of “Innovation Nation.”

So far, most of the companies using prediction markets are doing so in limited ways, in one or two departments, testing the concept to see how it goes. But in the last few years, corporate experimentation has moved beyond high-tech businesses into other industries, including retailing, consumer packaged foods, hotels, health care, steelmaking and telecommunications.

Today, analysts say, there are dozens of major corporations testing these markets. The companies include Google, Cisco Systems, GE Healthcare, General Mills, ArcelorMittal, the world’s largest steelmaker, and Swisscom, a large telecommunications company.

At the same time, a network of software and service suppliers is developing to cater to corporate prediction markets. The vendors include H.P. and smaller specialist firms like Consensus Point and NewsFutures. The field is attracting start-ups as well.

In 2006, for example, Adam Siegel and Nate Kontny left Accenture, the consulting firm, to found Inkling Markets. Mat Fogarty had been director for financial planning at Electronic Arts, where he experimented with prediction markets, before founding Xpree last year.

“Prediction markets are starting to move into the mainstream, and they will really change the way companies are run in the future,” said Emile Servan-Schreiber, the chief executive of NewsFutures.

At InterContinental Hotels, Zubin Dowlaty, vice president for emerging technologies, decided to create an online market last fall to “harvest and prioritize ideas” from within the hotel’s 1,000-person technology staff. “We wanted to tap the creative class that may not be able to voice their ideas,” Mr. Dowlaty said.

With InterContinental’s prediction market, players were asked to submit ideas anonymously, with a description and the benefit to customers and company. The bettors were given virtual tokens, each receiving 10 green ones to be placed on the best ideas and three red for bad ideas.

There were no limits on the number of times bettors could change their wagers as new ideas came to market, and the market was open for four weeks. The five top ideas (most green tokens), five bottom ideas (most red) and the top five bettors (most accurate, according to market consensus) were listed regularly.

The winners got $500, while second- and third-place finishers received $250 each. The winners, Mr. Dowlaty said, were engineers, analysts and contractors, not managers.

More than 200 people participated, submitting 85 ideas. One person proposed bringing back quarter-operated vibrating beds. “That one got beat down really fast,” Mr. Dowlaty said.

The winning ideas were suggestions to improve searching the company’s Web site to find and book hotel rooms. Two projects have been started as a result of the market, Mr. Dowlaty said.

Next, he said, prediction markets may be opened up to InterContinental’s customers, probably beginning with members of its Priority Club loyalty program. They could bet in markets for improving service and offerings, with points redeemed. “It’s the next frontier and the natural progression for this,” Mr. Dowlaty said.

Setting up corporate prediction markets can be tricky. Public markets for presidential candidates will attract thousands of bettors, but a company may want to run a market only for people with expertise in a certain product or project. At Hewlett-Packard, researchers have been working on techniques and software to make even small prediction markets efficient.

“We want to reduce the wisdom of crowds to the wisdom of 12 or 13 people,” said Bernardo A. Huberman, director of the social computing lab at Hewlett-Packard. Among the techniques, he said, are preliminary tests to assess the “behavioral risk characteristics” of participants to shade predictions from people who are inherently risk seekers or risk averse.

Starting a year ago, a group in the purchasing unit at H.P. began prediction markets on the price of computer memory chips three and six months ahead. The prediction markets, Dr. Huberman said, were up to 70 percent more accurate than the company’s traditional forecasting models. The more accurate predictions, he said, can be used to finesse purchasing, marketing and product pricing decisions.

The H.P. research project has become a service offering called Brain, for Behaviorally Robust Aggregation of Information in Networks. The service is now used in pilot projects by H.P. clients that include Swisscom, which is trying it to predict demand for new services like Internet television on cellphones.

At GE Healthcare, Steven Linthicum, manager for advanced prototyping and innovation, has recently managed prediction markets projects that have generated ideas for software products for the hospital and health care market. Based on that work, patents have been filed and projects are under way.

“These markets bring not only ideas, but what your organization thinks of the ideas,” Mr. Linthicum said. “That’s what leadership needs to know.”

G.E. is also evaluating how broadly prediction markets could be used in the health care division, a $17 billion-a-year unit, and elsewhere in the company.

“We’ll know a lot more at the end of the year how much this becomes part of the decision-making process,” Mr. Linthicum said. “We’re at the crossroads right now.”

In the near future there may be a reliable predictor of how an ad campaign or new product will do in the marketplace: Check the stock price.

Masterfoods, Best Buy, Google, Nokia, Samsung and Microsoft, among others, have recently tested internal stock markets that are used to predict looming events and assess ideas.

The mechanisms, called predictive markets, don’t always exactly mirror the stock market, but they basically work like this: Employees and consultants bet money on possible outcomes. The outcome that most people bet on is considered the most accurate.

Such predictive markets appeal to companies that have embraced crowdsourcing and books like James Surowiecki’s The Wisdom of Crowds. The idea is that large groups of people tend to be more intelligent than small groups acting alone. The Pentagon even used a predictive market, in this case a stock exchange, to gauge the viability of various terrorist threats. But complaints by Democratic senators, most notably California’s Barbara Boxer (who called it “sick”), prompted the Pentagon to kill it in 2003.

Many in the business world have embraced the concept, though, and some are applying it to marketing and advertising. Consensus Point, a Nashville, Tenn., firm that works with Best Buy, General Electric, Nokia and Samsung, has been offering such predictive market software for about 13 years, but company president Dave Perry said there’s been a big spike in demand over the past two. “There’s really been an inflection point,” he said. “The momentum is just amazing.”

Masterfoods, meanwhile, used a predictive market for a pet food launch last year, said Emile Servan-Schreiber, CEO of NewsFutures, a Baltimore predictive market software firm. Servan-Schreiber said employees were shown videos of prospective campaigns and were asked to estimate the impact on sales. Those that came closest got a prize. “It’s at least as accurate as a panel and much cheaper and much more reliable,” said Servan-Schreiber. Reps from Masterfoods could not be reached. NewsFutures’ other clients include Pfizer, Lilly and Dentsu, the Japanese ad holding company.

Justin Wolfers, assistant professor of business and public policy at the University of Pennsylvania’s Wharton School in Philadelphia, said last year some of his students worked with Gap in a predictive market program in which the students tried to pick clothing items they thought would sell the best. Wolfers said the issue isn’t that predictive markets are so clever, but that, “What firms use otherwise is so stupid.”

One key difference between a predictive market and a focus group, Wolfers said, is that people in the former are betting on what they think will sell while those in the latter are usually asked about what they like. He added that predictive market programs do “surprisingly well” with fake money.

Best Buy has used predictive markets with fake money over the past six months and found them accurate. For instance, the company’s internal stock market managed to predict when the chain’s first China store would open (Dec. 28, 2006). Employees who wagered correctly won gift cards. Barry Judge, CMO of Best Buy, said he can “see a day” when the company could use the stock market to judge how new ad campaigns will fare, but that hasn’t happened yet.

Hewlett-Packard is considered an innovator in predictive markets and has used the technology since the mid-’90s. HP doesn’t use predictive markets to evaluate ad campaigns or new products, but HP rep David Berman said they could be used to do that in the future. Berman said the prize is usually a “nominal fee” like $100. Said Berman: “If you put any kind of money on the table, people take the game a lot more seriously.”

A little over a month ago, a group of Yahoo! executives faced an enviable challenge. More than 200 employees had just completed the company’s annual Hack Day, where small teams have 24 hours to create, refine, and then pitch new products or services that they hope Yahoo will bring to market. Judges picked their favorites, but the company needed to decide which ones merited further development.

In years past, that decision was left to Yahoo executives. But this time, the company wanted help from the rank and file. So it doled out wads of play money and encouraged employees to place bets on which products would fly with the public. It used a system customized by Yahoo principal research scientist David Pennock, one that works a lot like a stock market. Yahoo used the data to help identify which ideas could be made into more complete prototypes. “As a product organization, you hope to remove the burden of predicting from one product manager,” says Prabhakar Raghavan, head of Yahoo Research.

Yahoo’s not alone in using in-house markets to shift that predicting burden. At least two dozen other companies, including Microsoft and Google , are basing decisions at least in part on employees’ betting patterns, using a tool known as a prediction market.

HOW IT BEGAN.  One of the earliest prediction markets was the Iowa Electronic Markets, founded in 1988 by the University of Iowa, to guess the winners of presidential elections. In the 1990s, a few companies, including Hewlett-Packard, began to apply prediction market theories to corporate events.

Now they are gaining wider acceptance. Markets are being used to let employees weigh in on a range of matters, from product launch dates to sales growth. Companies such as NewsFutures and Consensus Point even offer software and services to help companies set up their own prediction markets.

If it sounds like gambling, to some extent it is—except corporate prediction markets typically deal in play money. While some companies hand out cash prizes to winners, most give nonmonetary awards. The markets are helpful because they aggregate knowledge from a wide base of participants, and they can give CEOs a better handle on employee thinking. Still, getting busy workers to play along can be a struggle, and companies are divided on whether markets are reliable enough to justify the costs and the setup hassles.

ARCELOR’S ATTEMPT.  In September, 2005, Arcelor, the largest steel producer in Europe and Latin America, began testing prediction market software from NewsFutures in its flat products division. The company wanted to add another method of predicting quarterly variations in sales volume and prices of a key product, hot rolled steel.

“This is a very different concept,” says Javier Llera, senior manager of market intelligence at Arcelor’s flat products division. Collectively, the 30-40 employees who play the market have accurately predicted volumes, but pricing is more of a moving target. “During the three-month period prior to the start of each quarter, the collective forecast improves,” says Llera.

In the Arcelor market, participants are asked a question, such as the expected quarterly change in price for hot rolled steel in a given region. Each employee makes a prediction and records their level of confidence. They are able to see colleagues’ entries.

They’re also allowed to change predictions as new information arises, and they score points based on the accuracy of projections. Those with the highest scores win prizes such as a certificate for a weekend at a luxury resort.

A “RADICAL NOTION.”  But can groups working in a system like that really make better decisions than a few well-informed individuals? It’s a radical notion for many in the corporate world, where CEOs and a handful of other executives are paid top dollar for presumed expertise.

And yet the idea is commonly accepted in the stock market. “Firms are comfortable with markets; it’s a bit puzzling why they don’t use them more often internally,” says Todd Henderson, a professor of law at the University of Chicago.

Experts say prediction markets can work well for companies because they give a voice to employees who might not otherwise speak up. The markets are particularly useful in areas such as consumer goods or technology, where change is rapid and companies need to adapt quickly or get left behind.

And when it comes to introducing new products, there aren’t many other reliable ways to predict success, some experts say. Computer maker Hewlett-Packard ran a series of internal prediction markets from 1996-1999, to better forecast computer workstation sales. “In six out of eight cases, it was more accurate than internal corporate forecasts,” says Kay-Yut Chen, a principal scientist at HP Labs.

MICROSOFT’S METHODS.  Microsoft, the world’s biggest software maker, says prediction markets can be more reliable than other methods when it comes to key product milestones. “There is no other established forecasting mechanism,” says Todd Proebsting, a director at Microsoft’s Center for Software Excellence.

Companies often rely on a manager’s estimates of how a project is going, but those aren’t always accurate. Since August, 2004, Microsoft has run 15 to 20 markets studying when a particular feature will work, when a project is ready for testing, or the number of bugs that will be reported in a piece of software in a given period.

Google also uses prediction markets to forecast product launch dates. In the past 15 months, Google has run more than 130 markets on events such as launch dates and new office openings. The company even runs markets on how often particular products are likely to be used.

Executives won’t divulge specifics. But they could use a prediction market, for instance, to uncover the number of 7-day active Gmail users. In all, more than 1,000 employees participate in Google’s prediction markets, competing for T-shirts, gift certificates and cash prizes.

PUBLIC INTEREST.  And companies don’t just rely on their own internal markets. Prediction markets that are open to the public yield data on a range of matters affecting business plans. These include presidential election outcomes, gas prices, real estate values, a film’s performance at the box office, and even the probability of a flu pandemic. “Bird flu is one such potential risk with massive economic consequences,” says John Delaney, CEO of Dublin (Ireland)-based Trade Exchange Network, the parent company of both Intrade and TradeSports.com.

Intrade and TradeSports.com, along with HedgeStreet, are among public prediction markets that require participants to use real money. Other markets, including the Hollywood Stock Exchange, Foresight Exchange, NewsFutures Exchange, and the World Economic Forum’s Global Risks market, use play money.

Stephen Roman, a currency analyst at Forex Capital Markets, started playing real-money prediction markets about eighteen months ago. He’s trying to stay ahead of the world events that can have a profound impact on foreign exchange rates. “I’d always followed news events closely and the idea that you could see the odds of something happening was fascinating,” he says.

A PREDICTION PAYOFF.  Roman says that prices in a prediction market better reflect the probability of something happening than, say, odds on the winner of a horserace, since bookmakers build their transaction costs into odds.

Two recent bets that paid off for Roman: He called the resignation of Harvard President Lawrence Summers, and he predicted that former Iraqi Prime Minister Ibrahim al-Jaafari would leave office. “It is fun and it’s a little more fast-paced than the stock market,” he says.

Whatever the outcome, companies are cautious not to give too much credence to prediction markets. “This is a new concept, which is essentially being researched,” says Bo Cowgill, an analytics specialist at Google, who started the company’s prediction market. While several Google senior executives look at the prediction markets with some regularity, they don’t manage the company with that information, he says.

POSSIBLE PROBLEMS.  A company considering tapping employees’ collective wagering will have to tackle any number of challenges. For starters, CEOs need to ensure that they really want to know what a market could reveal, such as the likelihood of missing a sales target or product launch date. “The risk of creating markets is that it’s not fudgible, and if you ask the question, then you better want to know the answer,” says Robin Hanson, a professor in the department of economics at George Mason University.

If a product or another project is behind schedule, and everybody knows it, employees could lose motivation. “Everyone jumps off a sinking ship,” says Justin Wolfers, a professor at the Wharton School at the University of Pennsylvania.

Companies could also face a disclosure dilemma. Would executives would need to share with investors insights gleaned from these markets? “Prediction markets could make everyone in the company an insider from the SEC [Securities — Exchange Commission] perspective, if you don’t manage it well,” says Google’s Cowgill. This is one reason Google doesn’t use prediction markets to forecast revenue.

PROVIDING INCENTIVES.  Then there’s the participation problem. “Prediction markets aren’t everybody’s cup of tea,” says Microsoft’s Proebsting. If the mechanics of the market aren’t clear when a participant goes to the Web site, there’s a big chance of losing a potential trader right away, he says.

To combat this, Proebsting does a step-by-step presentation for groups who haven’t used prediction markets before. Microsoft, like many other companies, finds that somewhere between 40% to 60% of those who are invited actually take part.

Google’s Cowgill tries to stir up enthusiasm for his markets with fun events, such as taking bets on which team would win the World Cup. His market accurately picked Italy as the winner. But more importantly, for him, once employees started trading in the markets, they moved onto other business-related ones.

In the absence of fun events, executives may find it tough to give busy employees enough incentive to play the market. That was the case at HP in the late 1990s. Chen says that, due to budget constraints, HP could only give an average incentive of $50 per person per event.

IMPROVING THE SYSTEM.  During its three-year trial, HP found that prediction markets offered an accuracy improvement over other forecasting methods, though in most cases the improvement was only a few percentage points. “The accuracy improvement was not high enough to be adopted,” says Chen. “You need to be a lot more accurate before it’s worth it to implement a new process.”

HP has since moved on to a new game that Chen has invented. It also aggregates information from employees, but does it in a way that’s not a trading scenario. Chen says this new method is much more efficient.

Meanwhile, Yahoo is just getting started. The company is considering using prediction markets in other parts of its business, such as sponsored search. Says Pennock: “It’s the best known method we have for aggregating opinions and providing forecasts.”