These are Jasmine and Pascal – you may already know them from other NZZ videos. They also listen to music on Spotify. Spotify remembers exactly which songs Jasmine and Pascal are listening to, which they skip or which they have included in playlists.
Spotify then uses this information to create personalized playlists. Like the “Mix of the Week” for example. Spotify analyzes a lot more than just the genres and artists you listen to. In this video, we look at three methods that Spotify uses, among many others, to analyze us in detail. All three methods are based on artificial intelligence.
The first method is “collaborative filtering”: This is the comparison of the musical tastes of different users. We explain how this works in a simplified way using Pascal and Jasmine. Spotify creates a “taste profile” based on the listening habits of the individual users. So it breaks down your own taste in music. For example, Jasmine likes different kinds of rock, but also some pop. Pascal also listens to a lot of rock, but he also often goes in the direction of folk.
With so-called “collaborative filtering”, Spotify now compares the profiles with each other. It finds some similarities, but the differences in the listening activities are more exciting. Because Spotify uses them to search for a song in Jasmine’s playlists and her saved songs that Pascal doesn’t know yet. Spotify suggests Tame Impala’s song «The Less I Know the Better» to Pascal. From the many overlaps in the music taste of the two, Spotify concludes that he might like the song. In reality, the «collaborative filtering» compares the «taste profiles» of thousands of users who have a similar taste in music as Pascal. So the song suggestion isn’t just based on Jasmine’s taste in music. But “Collaborative filtering” is not the only method that allows the song to appear on Pascal’s playlist.
Spotify also does a meta-analysis. This technique is based on «natural language processing». The streaming service analyzes which words appear on the Internet in connection with the Tame Impala song. Spotify trawls through various websites, social networks and blogs. In our song example, words like “bass”, “psychedelic sounds” or “nostalgia” appear. We present our results here as a word cloud. This is then compared with the word clouds of other songs on Pascal’s playlists. The more agreement there is, the more likely it is that Spotify will suggest the song «The Less I Know the Better» to Pascal.
In the third method, Spotify takes a closer look at the song itself and its musical patterns. Every song consists of different frequencies. The frequency mix differs depending on the genre and the instruments used. For example, in an a cappella song, there are more high frequencies than in deep house.
His preferred average frequency mix is now determined from Pascal’s favorite songs. It is difficult for a human to understand exactly which patterns the algorithm filters out. After Spotify determines Pascal’s average frequency mix, it matches that mix to the specific frequency mix of The Less I Know the Better. There’s some overlap, and chances are the song fits Pascal’s musical tastes. So the result of all three methods is: Spotify suggests Tame Impala’s song to Pascal. Spotify recommendations use far more methods than the three we’ve shown in this video.
Officially, Spotify does not release any precise information about its algorithms and processes – trade secret. However, upon our request, Spotify confirmed that they use collaborative filtering, natural language processing, audio analysis, and other methods to create personalized playlists. So Spotify doesn’t give us a detailed look into its algorithm, but at the same time we are an open book for Spotify. Because Spotify collects far more information about us than, for example, which bands we like to listen to at the moment. Spotify also knows where we log into the app from and thus deduces our everyday habits. So when we go to work and what songs we like to listen to while we’re at it, what songs we listen to in the morning for breakfast and what we listen to at night before we fall asleep. Spotify not only knows our taste in music better than we do, but also other habits and preferences better than we might like.