Netflix Prize May Have Been Achieved

MadAnalyst writes "The long-running $1,000,000 competition to improve on the Netflix Cinematch recommendation system by 10% (in terms of the RMSE) may have finally been won. Recent results show a 10.05% improvement from the team called BellKor's Pragmatic Chaos, a merger between some of the teams who were getting close to the contest's goal. We've discussed this competition in the past."

Well done!

[Slurpee]

Well done Bellkor.

But now the real race begins.

Now that the 10% barrier has been reached, people have 30 days to submit their final results. At the end of the 30 days, whoever has the best result wins.

This is going to be a great month!

Re:Do they keep the prize money?

[lee1026]

AT&T have committed to giving all money to charity. The person at yahoo developed his entry while working at AT&T, so I will be surprised if yahoo gets any of it.

Re:No info about the Netflix prize

[Reikk]

Background: The Netflix Prize is an ongoing open competition for the best collaborative filtering algorithm that predicts user ratings for films, based on previous ratings. The competition is held by Netflix, an online DVD-rental service, and is opened for anyone (with some exceptions). The grand prize of $1,000,000 is reserved for the entry which bests Netflix's own algorithm for predicting ratings by 10%.

Re:No info about the Netflix prize

[MrMista_B]

What, you didn't even read the /summary/?

I know, this is Slashdot, but 'some basic info about it' is /right there/.

Interesting

[coaxial]

I published a paper using Netflix data. (Yeah, that group [slashdot.org].)

It's certainly cool that they beat the 10% improvement, and it's a hell of a deal for Netflix, since it would have cost them more than a prize money paid out to hire the researchers, the interesting thing is whether or not this really advances the the field of recommendation systems.

The initial work definitely did, but I wonder how much of the quest for the 10% threshold moved the science, as opposed to just tweaking an application. Recommender systems still don't bring up rare items, and they still have problems with diversity. None of the Netflix Prize work address any of these problems.

Still, I look forward to their paper.

Re:No info about the Netflix prize

[bogjobber]

If the first sentence didn't explain it enough, perhaps you could RTFA.

Re:Well done!

[nmb3000]

Now that the 10% barrier has been reached, people have 30 days to submit their final results. At the end of the 30 days, whoever has the best result wins.

That's true, but like the story title indicates, the prize may have been achieved. From the contest rules:

The RMSE for the first "quiz" subset will be reported publicly on the Site; the RMSE for the second "test" subset will not be reported publicly but will be employed to qualify a submission as described below. The reported RMSE scores on the quiz subset provide a public announcement that a potential qualifying score has been reached and provide feedback to Participants on both their absolute and relative performance.

So the publicly available submission beat the 10% mark, but only by a narrow margin of 0.05%. The private submission must also have surpassed 10% for them to be considered a preliminary winner, otherwise the contest goes on (at least, this is my understanding). In any case, I agree that the outcome should be interesting.

Re:No info about the Netflix prize

[Korin43]

Except it doesn't mention what an improvement of 10% means (unless you know what RMSE means, which I don't).

Re:No info about the Netflix prize

[JohnnyBGod]

Explanation of what RMSE is. [lmgtfy.com]

Re:Film recommendations

[coaxial]

I believe that Netflix is still using Cinematch. You could look into movielens [movielens.org]. It's from the GroupLens group at U Minn.

[E]ven though both probably use a pretty simple algorithm compared to Nextflix.

You do know that Netflix said on the outset "You're competing with 15 years of really smart people banging away at the problem." and it was beat in less than week [slashdot.org].

That's not to meant as a knock against Netflix's engineers, but more about that they didn't really build a state of the art recommender system. Simple SVD [sifter.org] (aka latent semantic indexing [wikipedia.org] outperfomed them as well.) They did something a bit more than straight up kNN clustering, but that was pretty much it.