Six Marketing Lessons of the NetFlix Crowdsourcing Experiment
Monday, September 28, 2009 at 9:55AM Here we follow up on the marketing highlights of last week's post on the $1 Mn NetFlix Prize announcement.
Why did the NetFlix Prize attract so much attention?
Here's one reason.
At a time when many companies are still afraid of social media – here’s NetFlix crowdsourcing their engineering knowledge, offering up customer data, knowledge often safeguarded inside a company’s IP sanctum sanctorem.
It’s bold. As many other news sources and this blog reviewed last week, it’s probably the best use of crowdsourcing to date. And, to many, it’s scary.
What specific lessons does the NetFlix Prize teach that can deployed inside and, more importantly, outside, your own company? I see six lessons for companies. (BTW - what's scary about any of this depends on which, if any, of these lessons appears alien to your company culture.)
1. Share with your industry’s research community.
In order to improve their recommendation engine’s accuracy in predicting the movie likes of their subscribers, the starting point for the NetFlix contest was their offer of a precious data set: 100 million ratings from over 480 thousand randomly-chosen, anonymous customers on nearly 18 thousand movie titles.
To win and take home a prize, the NetFlix contest rules also specified
….. you must share your method with (and non-exclusively license it to) Netflix, and you must describe to the world how you did it and why it works.
If you look at Wikipedia’s definition for the open source process, we see from the first few paragraphs that NetFlix pretty much followed it. (You only need substitute the word “database” for “source code”) In the full open source tradition, NetFlix didn’t just sequester the submitted solutions for their own company use – they fed it back into the community knowledge base.
2. Maximize the number of prizes you give away.
In many ways, NetFlix really offered many prizes to many different shades of stakeholders. (BTW- here's a key concept in crowdsourcing. Broaden your definition of who is a company stakeholder.)

Note several of these "prizes" are free, posing no additional cost of goods for a company. (This doesn't mean some of these prizes, such as the customer data set, aren't tricky to manage, as NetFlix' current vexations with consumer privacy advocates now show.)
3. Keep them motivated for the long haul.
Tough challanges like solving a design problem, optimizing a movie database recommendation system for subscribers, can take months or years to make progress. In fact, the more valuable the problem being solved is to your company, the longer it will take.
The answer: Milestone or Progress Prizes. Per the NetFlix contest rules ...
Serious money demands a serious bar. We suspect the 10% improvement is pretty tough, but we also think there is a good chance it can be achieved. It may take months; it might take years. So to keep things interesting, in addition to the Grand Prize, we’re also offering a $50,000 Progress Prize each year the contest runs. It goes to the team whose system we judge shows the most improvement over the previous year’s best accuracy bar on the same qualifying test set.
Complementing the Progress Prizes, NetFlix offered a leaderboard for the contestants and media to watch.
Why is this gaming approach a key strategic element? Not only did this give contest entrants a sense of momentum, more significantly, it allowed NetFlix to incorporate incremental improvements into their product along the way.
4. Use competitive energy. Build in a “Last Call”.
After each prize submission, NetFlix made a public announcement that allowed all other teams to submit a better alrogithm within a 30-day window.
(For an idea of the sporting drama this lent to the final days and even minutes of the competition between BellKor’s Pragmatic Chaos and The Ensemble teams, check out Wired’s piece.)
Marketing-wise, this “Last Call” phase resulted in heightened PR and media attention to the contest, with the top tier media and techno blogosphere all tracking the contest’s progress with bated breath.
5. Keep the attention of the virtual talent --- even when it’s all over.
To maintain the momentum (and perhaps preempt advances of companies seeking use of the newly identified talent) NetFlix launched a new contest on the very same day of the award announcement.
6. Warning: This new marketing territory may be No Country for Old Men
Okay - this one's definitely scary.
It’s a new day when open systems, crowdsourcing and bold marketing combine.
Are the rules changing?
NetFlix along with the many others bravely deploying crowdsourcing indicate they are changing.
Too often marketing and engineering are separate silos within a company. What NetFlix did here breaks across several silos, brilliantly interweaving a marketing initiative in and out of the R&D process, while also outreaching to the wider research community – giving them a virtual engineering department of 51,000.
For those afraid of open source process as putting their IP at risk – perhaps its time to think of the risks of not doing it, namely, much longer product cycles than your competitors.
As in the movie No Country for Old Men, it really pays to know in advance what you're looking at: Is it an oxygen tank or an air compressor used as a weapon? So too could be crowdsourcing in the hands of your competitors. PH...SSSST!
Useful new articles from others
ComputerWorld: Crowdsourcing takes Center Stage at DEMOfall 09
EbizQ Crowdsourcing: Five Reasons it’s Not Just for Startups Anymore
Reader Comments (4)
#7 Make a truly interesting, reachable problem. It doesn't sound like much, but the 10% increase in relevance that Netflix set as its target is a hard thing to do efficiently. It is also an interesting problem to crack. I doubt that many great talent would have been attracted to this contest if the problem had been to "sort the results by the size in bytes of the movies recommended" or to "double the relevance" of its search results--one is trivial, the other beyond the reach of current technology.
BTW, your #2 point is a very astute observation, missed by other writers who have analyzed this contest.
Thanks for the comments, Matt.
You're absolutely right on #7. Defining an intriguing, lofty, but not impossible problem was part of the genius of what NetFlix did.
Thanks for the compliment on #2. Unfortunately, in my experience, many marketing campaigns underestimate the amount of "free" (but highly valuable goods in others eyes) a company has to give away in trade to entice collaboration. So much more partner leverage comes about when you stop to inventory all the value directly or indirectly in your company's control.
I received your article via a Twine e-mail update. Very good analysis of the lessons learned from this crowdsourcing experiment. Similar to you I have been following this as it has progressed. In the end I am happy to see it worked out for everyone - Netflix, the winning team, as well as the markets in general. Perhaps other companies will take note and take a few risks themselves!
Thanks for your comments!
Yes, risk is the issue. This 3-year campaign was not without significant involvement of Netflix legal, marketing as well as a highly flexible engineering department. I am appreciative that smaller companies cannot bear these resources to take this type of risk,