Search the IBA site

Chronological Listing of Updates

Home
Up

 

The Netflix Prize

        16 Oct 06

 

Reed Hastings, Netflix CEO

 

 

“The beauty of the Netflix Prize is you can be a mathematician in Romania or a statistician in Taiwan, and you could be the winner.”

Reed Hastings, Chief Executive of Netflix, in The New York Times, 2 October 06

 

 

On 1 October 2006, Netflix, the DVD subscription service, announced a new contest centered around technical innovation.  A $1 million prize will go to the individual or group that can achieve a 10 percent improvement in the accuracy of movie recommendations based on personal preferences.

 

This is a really difficult challenge, as computer scientists and statisticians have been working on this problem for fifteen years.  To help contestants, Netflix is providing all registered users with a very large database of 100 million of its customers’ movie ratings.  They can use this database to test their new approaches. There’s also an objective scoring system that allows contestants to compare the performance of their system against that of Netflix’ existing approach, called Cinematch.

 

Netflix plans to keep the contest open for 5 years, to 2011.  Just 10 days after the contest was opened, there were already 9000 teams participating.  The contest runs a leaderboard, and has already received more than 150 valid submissions. Most of these performed worse than Netflix’ existing system.   However, the current leading team, “The Thought Gang,” had achieved a 1 percent improvement over the existing Netflix approach, which is already enough to qualify for a $50,000 “progress prize” if no one gets the whole 10 percent improvement this year.

 

 Netflix will not have ownership of the winning entry.  The winner of the Netflix Prize can sell the approach to Amazon, Blockbuster, Travel Advisor, or any other internet service that provides recommendations.  The winner is obligated, however, to license the approach to Netflix with a nonexclusive license.

 

The Netflix prize comes after the success of other high profile innovation contests.  Recently, for example, Stanford’s Artificial Intelligence Lab won the $2 million prize in the DARPA Grand Challenge, sponsored by the US government’s Defense Advanced Research Projects Agency.   Their robotically controlled vehicle, a modified VW Touareg, beat out about 40 competitors, from institutions like Carnegie Mellon and Cal-Berkeley, on a desert course in Nevada and California.  In May, 2006, DARPA launched the “Urban Challenge,” a contest in which robotically controlled vehicles have to navigate city streets, complete with pedestrians.

 

Building on DARPA’s example, the US National Aeronautics and Space Administration (NASA) recently unveiled a range of 13 different development contests, which it calls Centennial Challenges.  The contests range from building equipment like solar sails and lunar excavators to designing an “astronaut glove” that can be used in outer space for long periods of time.

 

In contrast to these government contests, the Netflix Prize has a much more open feel to it.  There are forums for contestants to discuss approaches, and a continuous updating regarding both the number of submissions and the best performance of these submissions.   Also, the costs of entry are much lower – Netflix provides the data, and all a contestant really needs is a computer, time, and the genius of a better-performing approach. 

 

 

 

The Leaderboard of the Netflix Prize – 10-12-06

 

Innovation contests, which bring in outside help to solve tough problems, are another example of the emerging practices around open innovation.  As Berkeley Professor Henry Chesbrough noted, companies begin to look outside for solutions when they realize that “not all the smart people work for them.” 

 

Netflix not only recognizes this, but also provides mechanisms that allow smart people from outside the company to find each other, get together, and try to improve Netflix’ performance. After all, if recommendations get better, more DVDs get rented.

 

Do you have a particularly thorny technical problem you’d like to solve in your business?  Have you considered sponsoring a public contest to find the solution?  Drop me a line and let me know.

 

More Information:

 

  1. Here’s the home page for the Netflix Prize.
  2. An October 2, 2006  article by Katie Hafner in The New York Times describes the Netflix Prize.
  3. I did an earlier update on the DARPA Grand Challenge in November 2005.  That’s here.
  4. Henry Chesbrough’s new book, Open Business Models, will be published in early December, 2006.  Here’s a link to it at Amazon.
  5. An overview of NASA’s Centennial Challenge is here.

 

 

 

Comments:

Eric, did you know that Elaine Allen has a Babson team working on this?

 

nice illustration of purpose driven contests, and efficient use of prizes.   These

are much more effective, when well designed than retrospective awards for

generating interesting applied work.   Retrospective awards, like the nobels,

do better at generating enthusiasm among the young for basic work, and 

retrospective awards can also work well in focusing attention on an issue,

or bringing innovations in a field that have public goods qualities to the attention

of broader communities.   I've actually designed, or co-designed, contests that have operate

in each context:  A Boston Fights Drugs Advertising Contest 20 years ago, which

changed the policy debate on how to communicate with kids about drugs, the 

Profiles in Courage Awards Offered by the JFK Library, The Innovations Awards

at the Kennedy School, and the Dively Award for Social Responsibility, also at

the Kennedy School.   It's great to see a program like this, because so many prizes

are simply wastes of time and money.

 

Great ICE Update this morning.  It’s an area I’m interested in, and I’ve been thinking about pulling together something about the history of “open source innovation.”  We haven’t used the process ourselves at Tapestry, but I’ve done some research on it and have come across some numbers and facts that you might be able to use:

 

There are three big companies that are acting as brokers for this.  Companies can anonymously post research problems that they are facing, and individuals can propose solutions.  Check out InnoCentive (www.innocentive.com), yet2.com (www.yet2.com) and Nine Sigma (www.ninesigma.com).  Chris Reidy, in the Boston Globe, did an interesting piece on this on 21 August called “10,000 Heads are Better Than One”  (http://www.boston.com/business/globe/articles/2006/08/21/100000_heads_are_better_than_one).  Here are two snippets from the article:

 

This posting caught Ed Melcarek's eye during one of his regular visits to InnoCentive.com, a Web site that offers bounties of up to $100,000 for solving scientific problems: Devise a more efficient way for getting toothpaste ingredients into a tube.

 

The Canadian engineer sent an e-mail that suggested putting a positive charge on fluoride powder and then grounding the tube. Colgate-Palmolive Co., the InnoCentive client that had posted the problem, liked the idea, and he earned $25,000 for a few hours of work.

 

"It's a beautiful way of doing business," he said.

 

…and…

 

Another solver identified by InnoCentive was David Bradin, a North Carolina patent attorney with a master's degree in organic chemistry. In 2002, he spotted a challenge seeking a more cost-efficient way to mass-produce a certain acid. Based on a reaction he had observed as an organic chemist, Bradin thought he knew the answer. A few months later, InnoCentive told him his e-mail solution had been accepted.

 

"It was the easiest $4,000 I ever made," he said.

 

Bradin never learned the identity of the company.

 

A Procter & Gamble Co. executive described how a North Carolina patent attorney solved one of the company's challenges.

 

"He's a lawyer by day and a chemist by night," P&G's Larry Huston said. "He did chemistry while his wife read romance novels."

 

Of course, this isn’t new.  In 1919, a gentleman named Raymond Orteig offered a prize of $25,000 to the first person who could fly an airplane non-stop between New York and Paris. The Wright Brothers had left the earth 16 years earlier, but flight hadn’t grown rapidly.  Certainly not to the extent that we see today.  Orteig thought it was time for that to change.

 

In 1927, eight years after Orteig offered his reward, Charles Lindbergh climbed into the cockpit of the Spirit of St. Lewis and flew solo for 30 hours. But he wasn’t the first to try—Hubert Julian, Rene Fonck, Charles Nungesser, Francois Coli and others tried and failed. Nine teams in total spent around $400,000 to try and win a prize that was 1/16th of that amount (hardly cost effective.)  But Lindbergh succeeded. He promptly became a national hero. Air flight took off. The value of aviation companies skyrocketed. And on 16 June 1927, Lindbergh collected the $25,000 prize personally from Orteig.  I’ve attached a photo from that meeting.  Look at how happy Orteig is to part with his money! 

 

Seventy years later, in May 1996, Peter Diamandis created the X PRIZE for the first group that could launch a ship into space (62+ miles up), and then repeat it within two weeks. The prize eventually grew to $10 million. Just like with Orteig, it took eight years before someone claimed it. But, in October 2004, Mojave Aerospace Ventures flew SpaceShipOne for the second time in two weeks, winning the award.  The comparisons to the Orteig Prize don’t end there. The cost to build and fly SpaceShipOne? Estimated at about $30 million, three times larger than the prize. Again, hardly cost effective. But like the Spirit of St. Louis, SpaceShipOne might be standing at the doorstep of a new industry: Richard Branson and Virgin Galactic will be offering space flights based off of the SpaceShipOne model as early as 2008.

 

Lindbergh crossed the Atlantic Ocean in 1927. That year:

bullet Applications for pilots licenses in the US increased 300%
bullet The number of licensed aircrafts in the US increased 400%
bullet Airline passengers went from 5,782 in 1926 to 173,405 in 1929

 

When we look back on SpaceShipOne in two decades, what numbers will we be quoting? Will we be amazed that “back in 2006,” getting into orbit wasn’t as easy and common an occurrence as crossing the Atlantic is today?  I think so.

 

PJ

 

 

 

Creative Commons License

All content on this site licensed under a Creative Commons License.