Large-Scale Recommendation Systems

Recommendation systems are ubiquitous in e-commerce and internet applications. However, most recommender systems have been tuned to achieve low error rates for straightforward prediction tasks (predict the rating for a movie, predict whether the item will be purchased, etc.)  —  e.g., the Netflix Prize, but in many domains, recommendation systems forms a multidimensional problem with several constraints.

We have been looking at some of these problems from the perspective of course recommendations, for the CourseRank project at Stanford. (CourseRank was launched as a startup company in 2010, and then acquired by Chegg. The CourseRank technology is being used at over 500 universities in the United States.) Course recommendations poses an especially tricky problem, because there are several dimensions involved.

Firstly, there are the problem of requirements. Every university program stipulates that in order to graduate the student needs to satisfy a set of requirements. These requirements could read like:

take 2 courses from {a, b, c, d}, but b and c together don't count.

Next, we looked at the problem of prerequisites. For example, the course linear algebra needs to be taken before calculus. When we make recommendations, we need to make sure that we recommend a package of items such that the prerequisites are present in the package itself. In general, we could have a graph of prerequisites, with OR (take either a or b to be eligible to take c) as well as AND prerequisites.

Third, people have looked at sequence mining with the aim of generating rules (e.g., a purchase of Diapers implies a purchase of Beer), but no one has used these ideas to form an aggregated recommendation (again, highly relevant for courses). We use probabilities and approximations to solve the problem of temporality in recommendations.

Lastly, in a viewpoints article at CACM, we suggested how new companies can significantly benefit from designing a combined back-end infrastructure to support search, recommendations and advertising, and how the three technologies are rapidly converging; differing if at all on a small number of aspects.