- [Narrator 1] With 500 million monthly users,
Spotify is the world's largest music streaming service.
- [Narrator 2] Spotify is the home of audio.
- [Narrator 1] It's known for its personalized playlists,
made with its recommendation algorithm.
- Think about users as this raw material,
and then on top of the data layer,
we're able to build shared models.
- [Narrator 1] But relying so much
on artificial intelligence has also drawn criticism
from some industry experts worried about algorithmic bias.
Here's how Spotify uses AI
to personalize users experiences on the platform.
This is the tech behind Spotify.
In the early two thousands,
many people found music recommendations through top charts
and early streaming platforms like Pandora and Last.fm.
- [Narrator 3] With the Last.fm app from the App Store,
you can listen to great bands.
- [Narrator 1] So, when Spotify entered the scene in 2008.
- It's not so much that they the first people
to start using analytics to recommend music,
but it was the way
in which they combined various computational techniques
in order to make their recommendations feel more lifelike.
- [Narrator 1] Thomas Hodgson studies algorithms
and artificial intelligence
with a focus on how new technology
from music streaming companies impact artists.
- Fans who have listened to Discover Weekly
and Daily Mix.
The way that they talk about them
is in very human-like terms.
- [Narrator 4] Discover Weekly, you magnificent.
You've done it again.
- [Narrator 1] In 2014,
Spotify acquired music analytics firm The Echo Nest,
which blended machine learning
and natural language processing
to build a database of songs and artists.
Spotify says this technology marked an important step
in the evolution of its recommendation system.
So, how does that system work?
It starts with a process called collaborative filtering.
- Collaborative filtering looks at the pattern,
across all of this data and tries to understand
when do tracks happen to be playlisted together very often?
You can think of it as building a map of music and podcast.
- [Narrator 1] That map looks something like this.
Each point represents a different track
in Spotify's catalog
and the location of each point
is determined by collaborative filtering.
- Which means that these tracks go together,
according to the way users have playlisted them
and listened to them.
- [Narrator 1] So, if these two songs,
are frequently playlisted together
they will be close to each other in this map.
Whereas if these songs are never playlisted together
they will be farther apart in the map.
But recommendations based purely
on collaborative filtering aren't perfect.
For example, during the holidays,
Mariah Carey's "All I want for Christmas is You,"
might get playlisted more frequently with "Silent Night,"
Even though this sounds like a pop song.
♪ Baby all I want for Christmas is you ♪
And this sounds like a Christmas Carol.
♪ Silent night ♪
If Spotify only generated recommendations,
based on proximity,
then users who like Mariah Carey,
might get recommended "Silent Night"
when they aren't interested in Christmas Carols.
To prevent this,
Spotify adds another layer of analysis,
called content-based filtering.
This algorithm gathers metadata like the release date
and label and executes a raw audio analysis.
It uses metrics like dance ability and loudness
to describe the sonic characteristics of the track.
These are the results for "Uptown Funk"
which sounds like this.
♪ This hit, that ice cold ♪
♪ Michelle Pfeiffer, that white gold ♪
And has the Dance Ability score of 0.856
on a scale of zero to one.
The algorithm also dissects each track's temporal structure.
Here's the visual representation
of that for "Anti-Hero" by Taylor Swift.
These are the beats, the bars and the sections.
Content-based filtering also takes
into account attracts cultural context
which means studying the lyrics
and analyzing the adjectives used
to describe the track in articles and blogs.
These filtering techniques are not unique to Spotify,
but industry experts say what sets the platform apart
is the amount of user data it has
and the products it creates from it.
Spotify says its content based filtering technology
has evolved over the years
and now includes more advanced proprietary facing features.
(upbeat music)
But Hodgson says the danger with algorithms
is that they could reinforce existing biases.
- This could mean that a particular catalog of music
has more male artists than female artists.
One of the dangers with machine learning
is that as listeners start to engage
with that catalog those biases become magnified
and that this creates what's called a kind of feedback loop.
- [Narrator 1] Spotify says it's research teams evaluate
and mitigate against potential algorithmic inequities
and harms and strive for transparency about its impact.
Another criticism is that the algorithm isn't optimized
for new artists because there's no user data.
This is known as the cold start problem.
Sultan says this is where human editors,
play a significant role in delivering recommendations.
- They're possibly some of the best people in the world
at trying to understand new releases
and culture and what's relevant.
- [Narrator 1] But Hodgson says,
the bigger concern is that certain metrics used
in the platform's audio analysis might be culturally biased.
- In other parts of the world,
they have musical systems and musical cultures
that are entirely different.
- Like this North Indian classical track for example.
(gentle music)
Spotify's algorithm labels its key signature as E minor
which Hodgson says is inappropriate
for this musical tradition.
- However, it's still the case that the music
that is emerging from South Asia
is being understood algorithmically,
under the Western equal temperament scale.
- [Narrator 1] Spotify says the audio analysis
is one small part of the overall system,
which takes into account many factors,
before making a recommendation.
Some industry experts also point to issues
with how the system understands metadata
for classical music.
For example, the metadata for a Tchaikovsky track,
can include not just the name of the work and the artist,
but also the movement, opus number and conductor.
Spotify's algorithm isn't optimized for that.
Apple Music, which has emerged in recent years
as a competitor to Spotify,
released a new app in March
that the company says is designed to solve this problem.
(gentle orchestral music)
Spotify says it doesn't comment
on a competitor's marketing campaigns.
In February, the streaming service joined the recent buzz,
around generative AI.
- [X] I'm X, and from this moment on
I'm gonna be your own personal AI DJ on Spotify.
- [Narrator 1] The DJ gives the algorithm a human voice
and offers listeners additional context,
around a recommendation.
- [X] Up next,
I know you've been on a summer song kick lately.
- [Narrator 1] Sultan says,
the company is also exploring reinforcement learning,
a technique that would allow the recommendation system
to learn automatically based on feedback.
- It will help with the diversity of their recommendation,
it will help with the longer term retention
and we are trying to push the state-of-the-art
in each of those, introducing new technologies,
new capabilities and bringing new experiences.
(upbeat music)