The crowd is wise (when it’s focused)

 Few concepts in business have been as popular and appealing in recent years as the emerging discipline of “open innovation.” It is variously described as crowdsourcing, the wisdom of crowds, collective intelligence and peer production — and these terms apply to a range of practices.

 The overarching notion is that the Internet opens the door to a new world of democratic idea generation and collaborative production. Early triumphs like the Linux operating system and the Wikipedia Web encyclopedia are seen as harbingers.

 In the new model, innovation is often portrayed as a numbers game. The more heads, the better — all weighing in, commenting, offering ideas. Collective knowledge prevails, as if a force of egalitarian inevitability.

 But a look at recent cases and new research suggests that open-innovation models succeed only when carefully designed for a particular task and when the incentives are tailored to attract the most effective collaborators. “There is this misconception that you can sprinkle crowd wisdom on something and things will turn out for the best,” said Thomas W. Malone, director of the Center for Collective Intelligence at the Massachusetts Institute of Technology. “That’s not true. It’s not magic.”

 The Netflix Prize is a stellar example of crowdsourcing. In October 2006, Netflix, the movie rental company, announced that it would pay $1 million to the contestant who could improve the movie recommendations made by Netflix’s internal software, Cinematch, by at least 10 percent. In other words, the company wanted recommendations that were at least 10 percent closer to the preferences of its customers, as measured by their own ratings.

 The front-runner is a seven-person team, and its members are statisticians, machine learning experts and computer engineers from the United States, Austria, Canada and Israel. It is led by statisticians at AT&T Research.

 The leading team is a very elite crowd, indeed, but it is also one that was made possible by the Internet. The original three AT&T researchers (one has since joined Yahoo Research, but remains on the contest team) made good strides in the first year of the contest. But to make further progress, they went looking for people with other skills and perspectives. So they reached out eventually to a pair of two-person teams, who were among the leaders in the rankings posted on the contest Web site.

 “The leader board was right there,” said Chris Volinsky, director of statistics research at AT&T. “It was pretty obvious who the top teams were.”

 Though leading, his team may not win. But the teams in close pursuit are similar collaborations of skilled researchers and engineers.

 The Netflix contest has lured experts worldwide not only because of the prize money but also because it offered a daunting challenge. The contestants’ algorithms must find patterns nestled in a collection of more than 100 million movie ratings.

 What is learned in tackling such a large-scale data analysis and predictive-modeling problem could well be applied in many industries, like Web commerce or telecommunications. “It made sense for us both from the perspective of AT&T and scientific research,” Volinsky explained.

 In the Netflix contest, the winning idea is simply the one with the highest score. But often, companies rely on a contributing crowd for ideas, though management then chooses. IBM, for example, conducts online brainstorming sessions it calls Jams — 13 over the last seven years.

 IBM used one session to guide its strategy for investing in new growth fields, starting in 2006. An estimated 150,000 employees, clients, business partners and academics participated. Management sifted through the ideas and committed $100 million to invest in several opportunities to apply technology innovations to energy-saving, health care and smart electricity grids.

 Open-innovation models are adopted to overcome the constraints of corporate hierarchies. But successful projects are typically hybrids of ideas flowing from a decentralized crowd and a hierarchy winnowing and making decisions.

  Opening the corporate doors to ideas and inspiration from the collective crowd holds great potential, but there are pitfalls, warns Henry Chesbrough, executive director of the Center for Open Innovation at the University of California, Berkeley. To succeed, Chesbrough said, a company must have a culture open to outside ideas and a system for vetting and acting on them.