Recommendation System we called is a very NARROW concept: A system that predicts ratings or preferences a user might give to an item. Rather than a machine learning algo that solve general recommendation problem.

Recommendation signal:

YouTube use the minutes you watch heavily in their recommend system.

Metrics to evaluate the recommendation system

RMSE, MSE, Hit rate, leave-one-out cross validation, reciprocal hit rate, cumulative hit rate

“There is more art than science in selecting the surrogate problem for recommendations”. Basically user behavior (A/B test) is always more important than accuracy.

YouTube said recommendation system in their paper

Offline accuracy (predicted rating vs real rating, then ranking) may not be the best way for recommendation. Turn on hit rate metrics or A/B testing should be more close to user experience.

Collaborative filtering is suitable for system with large number of clients and it is faster to run than general content-based filtering method.

Item-based collaborative filtering is generally easier/faster than user based

  1. Items tend to be more permanent nature than people. People’s flavor may change overtime
  2. Small number of items catalog compare to user -> less storage and faster
  3. Better experience for new users. Item can start give recommendation when user start to like some items, while user based recommendation system require new users to have enough behaviour.

One thought on “Building Recommendation System

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s