RecSys 2020, Key Takeaways — Part 2
Link to Part 1: RecSys 2020, Key Takeaways — Part 1
This is a continuation of my notes from my previous post on some of the sessions that I attended in RecSys 2020.
The video recordings of the paper presentations, tutorials and workshop are now available on Vimeo: ACM RecSys 2020 .
Selected Long Papers
Contextual and Sequential User Embeddings (Spotify)
The authors discuss the problem of modeling user context in recommending a sequence of songs to a user in a session. They propose a RNN based architecture which learns embeddings for a context within a session. They call their approach CoSeRNN and contrast/compare their approach against other state of the art models like LatentCross (used in Youtube), JODIE (Joint Dynamic User-Item Embeddings), RRN (Recurrent Recommender Network).
Performance estimates for augmenting recommendations ensembles (Best Paper Runner Up)
In an ensemble of base models , for each user and each base model, we can estimate the performance of each base model for a user from the respective user’s rating history. These performance estimates provide insight of which base model is best suited in modeling a user’s behavior. Using these performance estimates as extra features, improves the overall performance of the ensemble.
Neural Collaborative Filtering Vs Matrix Factorization Revisited (Google)
Neural Collaborative Filtering was originally proposed in WWW’17. This model learns two towers of neural networks (one each on item and user) and combines the final layers of the two towers using a multi-layer perceptron.
The authors reproduce the experiments in the original NCF paper and show that a simple dot product of the final layers of the two towers performs better than the MLP based NCF architecture. This is in contrast with the conclusions of the original NCF paper! Moreover, a MLP has more degrees of freedom to approximate/learn any function than a dot product.
The authors suggest that this is probably because the dot product function is difficult to learn with a MLP (needs a larger number of parameters to approximate element wise multiplication).
A case study of session based recommendations (Home Depot, Relational AI, Hyper Cube)
The authors show results from an interesting case study of session based recommendations systems on how offline metrics relate to the performance of a model from a human perspective. The authors experiment with 14 different models across the ML spectrum from deep models to shallow models and train them all on the same dataset offline. They evaluate the performance of the models on a variety of offline metrics like Precision, Hit Rate, NDCG, Coverage etc.
They pick the top 5 models which perform reasonably well as per offline metrics and send it for validation from human experts. Surprisingly, shallow models had better offline metric values than deep models. But the mis-predictions of the deep models were often more relevant than the mis-predictions of the shallow models. Human experts ended up giving better ratings to the deep models than the shallow models.
Further A/B testing confirmed that the deep model could improve user satisfaction. But the final deep model which was pushed to production was ironically not the best model as per any offline metric!
Tutorials, Workshops and more
Besides, long papers there were many interesting short papers/posters, tutorials and workshops presented at the conference.
Here are some (video recordings of) interesting tutorials presented at the conference:
- Conversational Recommender Systems
- Counteracting Bias and Increasing Fairness in Search and Recommender Systems
- Introduction to Bandits in Recommender Systems
- Counterfactual Policy-based Recommendation
- Adversarial Learning for Recommendation
Workshops: There were dedicated workshops for specific areas within recommender systems like Health, Podcast and Fashion. For areas on open research, there were workshops on context-aware systems, misinformation/harm-aware systems and fairness-aware systems. Since recommender systems is one of the main uses cases of multi armed bandit, there were some workshops around online learning, bandits and reinforcement Learning for user modeling. In this year’s RecSys, there was a Twitter Engagement Prediction challenge and a corresponding workshop on some of the best submissions.