A little over a month ago, a group of Yahoo! executives faced an enviable challenge. More than 200 employees had just completed the company’s annual Hack Day, where small teams have 24 hours to create, refine, and then pitch new products or services that they hope Yahoo will bring to market. Judges picked their favorites, but the company needed to decide which ones merited further development.

In years past, that decision was left to Yahoo executives. But this time, the company wanted help from the rank and file. So it doled out wads of play money and encouraged employees to place bets on which products would fly with the public. It used a system customized by Yahoo principal research scientist David Pennock, one that works a lot like a stock market. Yahoo used the data to help identify which ideas could be made into more complete prototypes. “As a product organization, you hope to remove the burden of predicting from one product manager,” says Prabhakar Raghavan, head of Yahoo Research.

Yahoo’s not alone in using in-house markets to shift that predicting burden. At least two dozen other companies, including Microsoft and Google , are basing decisions at least in part on employees’ betting patterns, using a tool known as a prediction market.

HOW IT BEGAN.  One of the earliest prediction markets was the Iowa Electronic Markets, founded in 1988 by the University of Iowa, to guess the winners of presidential elections. In the 1990s, a few companies, including Hewlett-Packard, began to apply prediction market theories to corporate events.

Now they are gaining wider acceptance. Markets are being used to let employees weigh in on a range of matters, from product launch dates to sales growth. Companies such as NewsFutures and Consensus Point even offer software and services to help companies set up their own prediction markets.

If it sounds like gambling, to some extent it is—except corporate prediction markets typically deal in play money. While some companies hand out cash prizes to winners, most give nonmonetary awards. The markets are helpful because they aggregate knowledge from a wide base of participants, and they can give CEOs a better handle on employee thinking. Still, getting busy workers to play along can be a struggle, and companies are divided on whether markets are reliable enough to justify the costs and the setup hassles.

ARCELOR’S ATTEMPT.  In September, 2005, Arcelor, the largest steel producer in Europe and Latin America, began testing prediction market software from NewsFutures in its flat products division. The company wanted to add another method of predicting quarterly variations in sales volume and prices of a key product, hot rolled steel.

“This is a very different concept,” says Javier Llera, senior manager of market intelligence at Arcelor’s flat products division. Collectively, the 30-40 employees who play the market have accurately predicted volumes, but pricing is more of a moving target. “During the three-month period prior to the start of each quarter, the collective forecast improves,” says Llera.

In the Arcelor market, participants are asked a question, such as the expected quarterly change in price for hot rolled steel in a given region. Each employee makes a prediction and records their level of confidence. They are able to see colleagues’ entries.

They’re also allowed to change predictions as new information arises, and they score points based on the accuracy of projections. Those with the highest scores win prizes such as a certificate for a weekend at a luxury resort.

A “RADICAL NOTION.”  But can groups working in a system like that really make better decisions than a few well-informed individuals? It’s a radical notion for many in the corporate world, where CEOs and a handful of other executives are paid top dollar for presumed expertise.

And yet the idea is commonly accepted in the stock market. “Firms are comfortable with markets; it’s a bit puzzling why they don’t use them more often internally,” says Todd Henderson, a professor of law at the University of Chicago.

Experts say prediction markets can work well for companies because they give a voice to employees who might not otherwise speak up. The markets are particularly useful in areas such as consumer goods or technology, where change is rapid and companies need to adapt quickly or get left behind.

And when it comes to introducing new products, there aren’t many other reliable ways to predict success, some experts say. Computer maker Hewlett-Packard ran a series of internal prediction markets from 1996-1999, to better forecast computer workstation sales. “In six out of eight cases, it was more accurate than internal corporate forecasts,” says Kay-Yut Chen, a principal scientist at HP Labs.

MICROSOFT’S METHODS.  Microsoft, the world’s biggest software maker, says prediction markets can be more reliable than other methods when it comes to key product milestones. “There is no other established forecasting mechanism,” says Todd Proebsting, a director at Microsoft’s Center for Software Excellence.

Companies often rely on a manager’s estimates of how a project is going, but those aren’t always accurate. Since August, 2004, Microsoft has run 15 to 20 markets studying when a particular feature will work, when a project is ready for testing, or the number of bugs that will be reported in a piece of software in a given period.

Google also uses prediction markets to forecast product launch dates. In the past 15 months, Google has run more than 130 markets on events such as launch dates and new office openings. The company even runs markets on how often particular products are likely to be used.

Executives won’t divulge specifics. But they could use a prediction market, for instance, to uncover the number of 7-day active Gmail users. In all, more than 1,000 employees participate in Google’s prediction markets, competing for T-shirts, gift certificates and cash prizes.

PUBLIC INTEREST.  And companies don’t just rely on their own internal markets. Prediction markets that are open to the public yield data on a range of matters affecting business plans. These include presidential election outcomes, gas prices, real estate values, a film’s performance at the box office, and even the probability of a flu pandemic. “Bird flu is one such potential risk with massive economic consequences,” says John Delaney, CEO of Dublin (Ireland)-based Trade Exchange Network, the parent company of both Intrade and TradeSports.com.

Intrade and TradeSports.com, along with HedgeStreet, are among public prediction markets that require participants to use real money. Other markets, including the Hollywood Stock Exchange, Foresight Exchange, NewsFutures Exchange, and the World Economic Forum’s Global Risks market, use play money.

Stephen Roman, a currency analyst at Forex Capital Markets, started playing real-money prediction markets about eighteen months ago. He’s trying to stay ahead of the world events that can have a profound impact on foreign exchange rates. “I’d always followed news events closely and the idea that you could see the odds of something happening was fascinating,” he says.

A PREDICTION PAYOFF.  Roman says that prices in a prediction market better reflect the probability of something happening than, say, odds on the winner of a horserace, since bookmakers build their transaction costs into odds.

Two recent bets that paid off for Roman: He called the resignation of Harvard President Lawrence Summers, and he predicted that former Iraqi Prime Minister Ibrahim al-Jaafari would leave office. “It is fun and it’s a little more fast-paced than the stock market,” he says.

Whatever the outcome, companies are cautious not to give too much credence to prediction markets. “This is a new concept, which is essentially being researched,” says Bo Cowgill, an analytics specialist at Google, who started the company’s prediction market. While several Google senior executives look at the prediction markets with some regularity, they don’t manage the company with that information, he says.

POSSIBLE PROBLEMS.  A company considering tapping employees’ collective wagering will have to tackle any number of challenges. For starters, CEOs need to ensure that they really want to know what a market could reveal, such as the likelihood of missing a sales target or product launch date. “The risk of creating markets is that it’s not fudgible, and if you ask the question, then you better want to know the answer,” says Robin Hanson, a professor in the department of economics at George Mason University.

If a product or another project is behind schedule, and everybody knows it, employees could lose motivation. “Everyone jumps off a sinking ship,” says Justin Wolfers, a professor at the Wharton School at the University of Pennsylvania.

Companies could also face a disclosure dilemma. Would executives would need to share with investors insights gleaned from these markets? “Prediction markets could make everyone in the company an insider from the SEC [Securities — Exchange Commission] perspective, if you don’t manage it well,” says Google’s Cowgill. This is one reason Google doesn’t use prediction markets to forecast revenue.

PROVIDING INCENTIVES.  Then there’s the participation problem. “Prediction markets aren’t everybody’s cup of tea,” says Microsoft’s Proebsting. If the mechanics of the market aren’t clear when a participant goes to the Web site, there’s a big chance of losing a potential trader right away, he says.

To combat this, Proebsting does a step-by-step presentation for groups who haven’t used prediction markets before. Microsoft, like many other companies, finds that somewhere between 40% to 60% of those who are invited actually take part.

Google’s Cowgill tries to stir up enthusiasm for his markets with fun events, such as taking bets on which team would win the World Cup. His market accurately picked Italy as the winner. But more importantly, for him, once employees started trading in the markets, they moved onto other business-related ones.

In the absence of fun events, executives may find it tough to give busy employees enough incentive to play the market. That was the case at HP in the late 1990s. Chen says that, due to budget constraints, HP could only give an average incentive of $50 per person per event.

IMPROVING THE SYSTEM.  During its three-year trial, HP found that prediction markets offered an accuracy improvement over other forecasting methods, though in most cases the improvement was only a few percentage points. “The accuracy improvement was not high enough to be adopted,” says Chen. “You need to be a lot more accurate before it’s worth it to implement a new process.”

HP has since moved on to a new game that Chen has invented. It also aggregates information from employees, but does it in a way that’s not a trading scenario. Chen says this new method is much more efficient.

Meanwhile, Yahoo is just getting started. The company is considering using prediction markets in other parts of its business, such as sponsored search. Says Pennock: “It’s the best known method we have for aggregating opinions and providing forecasts.”

You probably could use help making decisions. If so, the markets may be the place to turn.

That’s the argument of some economists, who believe in the decision-making prowess of free markets — specifically, in so-called decision or prediction markets.

The best known markets — those for buying and selling stocks or bonds — also tap the collective knowledge of investors to forecast the future direction of interest rates or the economy. But decision markets are made expressly for forecasting. For instance, the Iowa Electronic Markets, which let players bet on political contests, have repeatedly bested opinion polls in predicting election results. Similarly, the Hollywood Stock Exchange, where participants wager on movie box-office returns and possible Oscar winners, boasts more than a half million regular players.

Now, several companies are using software that sets up decision markets. These markets tap the collective knowledge of an organization by letting employees bet on future events, such as forecasting sales or betting on the most promising new product.

Early Returns

It’s still too early to tell how effective such efforts will be, but early users say these markets work as well as or better than

traditional forecasting tools. “We believe the technology has a lot of potential,” says Christina LaComb, a computer scientist in General Electric Co.’s computational-intelligence lab in Niskayuna, N.Y., which last summer ran a test to see whether the markets could help brainstorm new project ideas.

Using software from Consensus Point of Nashville, Tenn., GE researchers created a market where participants could bet on what they thought would be the most feasible project.

With the market, anyone could float “stock” in a proposed project; lab members began with a fixed amount of money, and then could buy shares in their favorite. The price of projects, which began at $50, rose and fell based on demand, with prices capped at $99. At the end of three weeks, the project with the highest price was declared the winner, and the winning team received financing to develop its idea. (A GE spokesman says only that the idea had something to do with artificial intelligence.)

To understand why these markets work as well as they do, consider the usual alternatives companies have for aggregating this sort of knowledge: committee meetings, polling, reports or focus groups. Meetings are often dominated by the person with the best arguments or most forceful personality, not necessarily with the best information. Sales quotas, budgets or other factors can distort or deter the accurate sharing of information.

In contrast, in decision markets, prices reflect all the information players have — data, opinions or intuitions — about the likelihood of some future event. While each player may have very little information, collectively they have a great deal. By rewarding winners, markets give players the incentive not only to share the information they have (by placing their bets on the favored outcomes) but to make an extra effort to gather more data.

The markets also empower participants in ways that meetings or surveys can’t. That’s because companies can set it up so that a broad array of workers take part in the markets. “Guys on the shop floor think this is terrific because they get to talk to the CEO,” says Justin Wolfers, a Wharton School assistant professor of business and public policy who has studied decision markets. “CEOs like it because they get to talk to the guys on the shop floor.”

Adoption of the tools is expected to grow now that a few software vendors have begun to promote marketplace systems to corporate clients. Corning Inc. is running a pilot project with decision-market software from New York-based NewsFutures Inc., to see how markets might be used in forecasting the liquid-crystal display market. O’Reilly Media Inc., Sebastopol, Calif., is preparing a test to see how well decision markets can predict the optimal print run for its specialized software manuals, using a Web-based marketplace from a Chicago start-up, Inkling Inc.

Monthly Printer Sales

Hewlett-Packard Co. was one of the earliest companies to test decision markets for internal forecasting. In 1996, researchers at H-P Labs wanted to see whether the markets could better forecast monthly printer sales.

About 15 to 20 players from different parts of the company, including product marketing and finance, participated in the market. Each was given a portfolio of cash and securities — representing a range of total monthly printer sales — and then had the opportunity to buy and sell the different securities. Once the actual monthly sales figure was revealed, the security that contained the actual sales was declared the winner and holders were paid a set amount; the other securities paid out nothing.

In a test over the next three years, the markets beat official forecasts in six out of eight tries. Still, the markets were time-consuming for participants and, researchers found, not as efficient as they could be. So to improve their accuracy, researchers have streamlined the markets to reduce the amount of time it takes to play from several days to minutes. Since 2002, some of H-P’s business units have been testing the revised markets.

Using markets to predict the future “is a sound idea,” says Kay-Yut Chen, a scientist at H-P Labs. “The only question is what game we should use.”

Another possible use of decision markets is to help companies forecast a single event — for instance, whether a new drug will win regulatory approval. Such a market might work like this: The company sets up a market where players can buy two contracts — one that pays out, say $100, if the drug is approved and one that pays nothing if it isn’t. Then participants buy and sell the contracts based on what they believe will be the outcome. So if the “will be approved” contract is selling for $60, that represents the market’s forecast that there’s a 60% likelihood that the drug will be approved.

The market also can be used to forecast a range of possible outcomes, such as the projected demand for laptop computers. Here, players choose from among contracts that represent different sales ranges — under one million, one million to two million, and so on.

–Mr. Totty is a news editor for The Journal Report in San Francisco.  Write to Michael Totty at michael.totty@wsj.com.