Recommendation System we called is a very NARROW concept: A system that predicts ratings or preferences a user might give to an item. Rather than a machine learning algo that solve general recommendation problem.
YouTube use the minutes you watch heavily in their recommend system.
Metrics to evaluate the recommendation system
RMSE, MSE, Hit rate, leave-one-out cross validation, reciprocal hit rate, cumulative hit rate
“There is more art than science in selecting the surrogate problem for recommendations”. Basically user behavior (A/B test) is always more important than accuracy.YouTube said recommendation system in their paper
Offline accuracy (predicted rating vs real rating, then ranking) may not be the best way for recommendation. Turn on hit rate metrics or A/B testing should be more close to user experience.
Collaborative filtering is suitable for system with large number of clients and it is faster to run than general content-based filtering method.
Item-based collaborative filtering is generally easier/faster than user based
- Items tend to be more permanent nature than people. People’s flavor may change overtime
- Small number of items catalog compare to user -> less storage and faster
- Better experience for new users. Item can start give recommendation when user start to like some items, while user based recommendation system require new users to have enough behaviour.