Have you ever wondered whenever you open up your Netflix profile, most of the times how they successfully recommend you TV shows or movies of your choice and you be like this...
 |
| Source: Google |
If we go through the timeline of Netflix, we can see that they started as a DVD rental company. Later when Netflix moved online video streaming business, they declared $1 Million prizes to the developer whoever can beat their existing algorithm Cinematch's accuracy by 10% at predicting customer's ratings for video recommendation. Starting from 2006, every year they continue to organize this competetion.
What's up with $1 Million Prize?
The rating prediction algorithm which won in 2006, they mainly used two algorithms Restricted Boltzmann Machines for Collaborative Filtering and Factorization meets the neighborhood: a multifaceted collaborative filtering model.
Score measurement is done on based on RMSE scale. Netflix's own algorithm uses straightforward statistical linear models with a lot of data conditioning which was able to achieve RMSE score 0.9514. The catch of winning $1M was you have to improve the score by at least 10%, i.e., to obtain 0.8572 on test data set. As long as no team is able to reach the score, no one will get $1M, but the annual winner will get to keep $50,000. However, to win the grand prize, the algorithm must improve 1% RMSE score of the previous year' winner.
One way to recommend user new movies or shows is to suggest what's trending for a given timestamp, but suggesting all users same movies at the same time is precisely opposite of personalization. Just based on rating prediction, recommending a user some video is not such a good idea, other pieces of information should be considered as well such as time to first play, sessions without a play, days without a play, the number of abandoned plays and more. So for more compact personalization, Netflix came up with the idea of the personalized homepage. As the term itself suggests, for each user, the homepage will be utterly personalized given that user's taste of plays and other metadata about the user(where the user is from, at what time user like to watch shows, for how long at a streak user like to resume plays, etc.).
Resultant videos for each row mainly come from an individual algorithm independently. Each row either tagged with some genre(e.g., Thriller, Mystery, Comedy) or some predefined category by Personalized Video Ranker(PVR) by Netflix. In a research paper by Netflix authors have mentioned: "PVR works better when we blend personalized signals with a pretty healthy dose of (unpersonalized) popularity.".
Most personalized recommendations come in the BYW(Because You Watched) row, where based on the user's previous play history they tend to get the recommendations. BYW row suggestions also take into the account various factors like how long users watched the show before user quit, at what time user tend to spend time on (e.g., weekend or weekdays), how frequently user manage to come back for the same genre, etc.
The result of each row generated by collaborative filtering techniques and above all the methods help Netflix to stream their 80% of hours by the user. There are various types of meaningfully named rows in Netflix, which rows to be presented to a given user that is also decided by standard collaborative filtering techniques.
For each character user starts typing, Netflix starts to recommend the user and recommendations depend on the user taste as well. For, e.g., if I start typing "go" maybe I am getting "GOTHAM" as the first result, and another user is getting "GOOD WILL HUNTING" as the first result in his homepage.
Score measurement is done on based on RMSE scale. Netflix's own algorithm uses straightforward statistical linear models with a lot of data conditioning which was able to achieve RMSE score 0.9514. The catch of winning $1M was you have to improve the score by at least 10%, i.e., to obtain 0.8572 on test data set. As long as no team is able to reach the score, no one will get $1M, but the annual winner will get to keep $50,000. However, to win the grand prize, the algorithm must improve 1% RMSE score of the previous year' winner.
How Netflix recommendation works?
In an interview, ex VP of innovations of Netflix said: "A very realistic vision is we should get to the point where you just turn on your Netflix app, and automatically a video starts to play that you're delighted with.". The current recommender system of Netflix is so efficient that about 80% of streamed content comes from recommendation and rest 20% comes from the user search.One way to recommend user new movies or shows is to suggest what's trending for a given timestamp, but suggesting all users same movies at the same time is precisely opposite of personalization. Just based on rating prediction, recommending a user some video is not such a good idea, other pieces of information should be considered as well such as time to first play, sessions without a play, days without a play, the number of abandoned plays and more. So for more compact personalization, Netflix came up with the idea of the personalized homepage. As the term itself suggests, for each user, the homepage will be utterly personalized given that user's taste of plays and other metadata about the user(where the user is from, at what time user like to watch shows, for how long at a streak user like to resume plays, etc.).
How Netflix homepage recommends you videoes?
Each user's homepage contains several rows(mostly around 40) and each includes at most 75 recommendations of shows. |
| Source: Google |
Most personalized recommendations come in the BYW(Because You Watched) row, where based on the user's previous play history they tend to get the recommendations. BYW row suggestions also take into the account various factors like how long users watched the show before user quit, at what time user tend to spend time on (e.g., weekend or weekdays), how frequently user manage to come back for the same genre, etc.
The result of each row generated by collaborative filtering techniques and above all the methods help Netflix to stream their 80% of hours by the user. There are various types of meaningfully named rows in Netflix, which rows to be presented to a given user that is also decided by standard collaborative filtering techniques.
What Netflix does about rest of the 20%?
Around 80% of the times Netflix will lead the user to a particular show, but for the rest of the 20% user tend to follow their own choice. They either go by searching for some genre, actor or any particular videoes which also requires its own set of algorithms. For the searching part, the user must type set of keywords in the search box, and Netflix will help a user to retrieve information based on the keyword provided by them. The retrieved result will obviously have a significant impact on the user's watching or rating history. |
| Source : Google |
References:
- https://en.wikipedia.org/wiki/Netflix_Prize
- https://dl.acm.org/citation.cfm?id=2843948
- https://medium.com/netflix-techblog/learning-a-personalized-homepage-aa8ec670359a
I know! I was so shocked. I once saw a show by Andy Yeatman on Netflix and then I got suggestions for more. This is a smart move and I loved it. I got to see so many shows by him after that. They were entertaining and educating.
ReplyDelete