Reinforcement Learning for Personalization at Spotify with Tony Jebara - 609

Causal Modeling

Causal modeling at Spotify plays a crucial role in understanding user behavior and improving recommendation outcomes. Tony Jebara explains how the company tracks user actions to build trajectories, which are sequences of actions leading to a stream, helping to clarify attribution paths 1. These trajectories are used in offline reinforcement learning to simulate user interactions and predict long-term rewards. Jebara highlights the challenge of breaking users out of repetitive listening patterns, or the "rabbit hole," to explore new content, which requires understanding the broader journey and multiple nudges 2.

It's several things in sequence, and it's pretty easy to lose the attribution when it's several sequences of actions that led to the reward, and the reward is delayed.

--- Tony Jebara

This approach allows Spotify to enhance user engagement by encouraging exploration beyond familiar playlists.

   

Survival Modeling

Survival modeling at Spotify is used to predict user retention and transitions across different states. Tony Jebara describes the use of beta geometric survival models to calculate the lifetime value of users, considering various states like subscription types and geographical locations 3. This multistate approach aligns with reinforcement learning principles, offering a nuanced understanding of user behavior. Jebara uses the analogy of coin and dice factories to explain how users' transitions are modeled, emphasizing the complexity of predicting user actions 4.

So, what we're doing for each user is trying to predict what kind of coin factory are you running as a user?

--- Tony Jebara

This sophisticated modeling helps Spotify tailor its strategies to enhance user retention and engagement.