How do Spotify and YouTube seem to know exactly what you want to listen to or watch next?

The answer

Modern recommendation systems like Spotify's Discover Weekly and YouTube's suggestion engine combine collaborative filtering (finding users with similar tastes), content-based filtering (analyzing audio features or video metadata), and deep learning models to predict preferences. Spotify's Discover Weekly, launched in July 2015, generates 30 personalized songs per user each Monday using a combination of collaborative filtering on 100 million users' listening histories and convolutional neural networks that analyze the raw audio spectrograms of songs. YouTube's recommendation engine drives over 70% of all time spent on the platform and processes hundreds of billions of user signals daily using a two-stage deep neural network architecture described in a landmark 2016 RecSys paper.