When you’re surfing through Netflix on the bus, the train, work, or in the comfort of
your couch at home —
how do you pick what to watch?
Is it a catchy title?
Maybe an interesting synopsis?
Perhaps it’s the unwillingness to watch another episode of The Office...
“Save Bandit!”
[mewling cry of a cat ]
Or just maybe, a particular piece of cover art speaks to your spirit?
But go on your friend’s Netflix account — if they aren’t still using your subscription
— and you may find the cover art there doesn’t interest you.
In fact, what you see might be completely different.
It's no accident — Netflix's thumbnails are all tailor made...
for you.
At its heart, Netflix is all about creating personalized experiences — or, rather, calculating them.
They’ll draw you in with curated trailers of upcoming releases, point out new episodes
from previously watched videos, and even gauge your interest in content via match scores.
And while streaming services are notoriously tight-lipped about sharing their viewership data...
Over the years, Netflix has shared glimpses into how their technology works.
According to internal studies, a typical viewer spends 1.8 seconds considering each title,
and Netflix believes it only has 90 seconds to get your attention
before you move onto another activity.
And among all the things that could catch your attention and make you watch a show–
or several—
Netflix found that the biggest influence were the thumbnails.
Humans are intensely visual creatures.
Our eyes move three to four times a second to process new information.
And because Netflix’s goal is to get your attention and hold it, the company puts a
lot of work into choosing every thumbnail you’ll see.
But before they can decide what image will show up on your account,
they have to sift through a ton of data.
An hour-long Stranger Things episode has almost 86,000 video frames.
To figure out which ones will make the best thumbnails, Netflix uses a pretty scientific
selection process called Aesthetic Visual Analysis, or AVA.
AVA is a set of tools and algorithms that search Netflix videos for the best images
and pull them out to create thumbnails.
The process can be broken down into two basic steps.
First is frame annotation.
A program analyzes every static video frame of a video,
and image recognition algorithms use information gleaned from each shot to create metadata.
The metadata is like an electronic fingerprint —
identifying characteristics unique to each video frame.
All of this info helps build a database of information that makes it easier to pick out
the best images for thumbnails later.
And to sort all this data, the company groups it into roughly three categories that are
key to identifying good images:
Visual — focusing on brightness, color, contrast, and motion blur.
Contextual, which documents face and object detection, motion, and shot angles.
And compositional — which focuses on visual principles
in cinematography, photography, and design.
The second step is a process called image ranking.
An algorithm uses the metadata to pick out specific shots that
Netflix has determined are the most attractive and clickable:
ones that aren’t blurry, have varied imagery, feature major characters, and don’t contain
sensitive or unauthorized branded content.
Then, finally, a creative team steps in to use the best images to design the thumbnail artwork.
But the process doesn’t stop there — Netflix still has to figure out which ones work best
for each user.
A/B testing is executed — again, again, and again.
You’ll regularly see changes in your thumbnails — based on your engagement with previous titles.
On the most basic level, let’s say you’re a fan of comedy, and you’ve watched a bunch
of stand-up specials.
When you search for Good Will Hunting, you may get a thumbnail with Robin Williams,
a famous comedian, and one of the movie’s main characters.
But people more into romance titles could be shown the cover art with the two leads kissing.
There are also regional differences.
From glancing at a show’s thumbnails across different countries, you could infer Germany
is more into abstract images, and US viewers may prefer clearly defined characters and story plots.
Though there are a considerable number of thumbnails to choose from, and lots of science
that goes into each decision, it doesn’t mean Netflix gets it right all the time.
One Twitter user found the cover art for, “Like Father” a movie starring Kristen Bell,
Kelsey Grammer, and Seth Rogen, had cover art that didn’t exactly match who
the lead actors in the film were.
It was also kinda odd to see on my own account, that the Catwoman thumb has an image of a
barely recognizable supporting actress Sharon Stone,
and Blade II had someone other than its star, Wesley Snipes, in the thumbnail.
Netflix is obsessed with A/B testing new features like video promos, intro skipping, and auto-playing
trailers — just as obsessively as they are with testing thumbnails.
Netflix wants everyone to watch more, so it’s unlikely they’ll stop doing these tests.
And if you don’t like being a guinea pig, there is the option to opt out.
But with binge-watching quickly becoming a national pastime, it’s likely that,
when Netflix asks you,
"Are you still watching?"
You will be.
Hey! Thanks for watching The Goods, and thanks to our sponsor American Express.
AmEx has a credit card feature that gives you choices for how to make payments big or small
called, "Pay It Plan It."
"Pay It" helps you reduce your balance by making small payments throughout the month.
And "Plan It" can help you split purchases over $100 up over time.
You can check it out at americanexpress.com/payitplanit
And thanks again to American Express, their support made this series possible.