- (laughs) Hello Insiders, I'm Todd and I'm here with Eric.
Eric, why don't you tell us about yourself?
- Hello everybody, my name is Eric and I'm
a product manager working with Todd in recommendations.
My main focus on YouTube is to work
on satisfaction with recommendations.
What we do there is we try to make sure people
are happy with the videos they watch from their homepage
or on their watch next feed.
- Right, yes, we've been working on satisfaction
together for a few years now and I'm also excited
to finally be talking more to creators about this
because it's really important to our recommendations.
So can you just kinda tell us a little bit more
about what satisfaction means?
- Our goal is to make sure people are happy
with both the videos they're recommended when
they open YouTube on their homepage and when
they're watching a video and they watch later
or their watch next feed.
And we also want people to once they've seen the video
be happy with the fact that they've watch it.
And what I mean by this is I may watch three videos
to the end, from beginning to end but that doesn't
mean that I felt the same about all of them.
And it's important for us as YouTube to try
and differentiate between the cases where you thought,
"Wow, this was awesome, I really,
"I'm glad that I watched it."
And those where as a user you're like,
"Well, you know, I was hoping that this delivered
"something that actually didn't occur."
And on the other side, for recommendations, we want
to help users find videos that they want to watch,
both new stuff as well as stuff
that you, they're already watching.
And we want to make sure that whenever showing
things that are like, that turn you off from YouTube,
that you're either offended by or really unhappy about.
- We know that watch time is actually a decent signal
for helping us understand whether someone
liked the video or not.
But then, we recognized that not all watch time
was equally valued by our viewers.
Satisfaction is our effort to help differentiate
when watch time isn't all equally valued.
So how do we know if watch time made a user happy
or if they were disappointing?
- Yeah, it's a great question.
That's actually a big part of what we do.
We try to find ways to understand how users feel
about the videos they're watching.
And we look at both features that are on YouTube.
Like, did they press the Like button?
Did they say they were not interested on a video?
And other signals like that.
And we also use things like surveys.
We actually, you may have seen this, we sent surveys
to users and we ask like,
"What did you think of this video?"
And in a personalized way, we use that to decide
how high a video shows up on your page or whether
we should even show that video to a specific user.
- What does this, what are the implications for creators?
- Yeah, so I would like to say, first, that as
always YouTube's goal is to help viewers find videos
that they'll enjoy watching.
And satisfaction is just a piece of that.
So as a creator what you should be thinking about
is creating content that is good for the audience.
So if you create content that's for your audience,
viewers are going to be happy with it and the system
is going to be recommending it to them.
- How should a creator know whether their content
is good for the audience?
- Yeah, so there's a couple of factors to this.
On one side, when creating the content, I would say
if you are creating content that is misleading,
that's clickbaiting thumbnails that are meant
to get people to click on them but then the content
is actually not delivering on that, you can assume
that it's less likely to be satisfying to users.
On the flip side, we are still working on this,
but we would like to provide more signals to creators
like we do for other things like, click-through rate
or average few duration so that you can get a better sense
of when a video is performing in one way or another
what your audience feels about it.
In the meantime, what I would say, you can look
at soft signals like, are people commenting
on the video and saying they're happy
with what you're creating?
And do you find that people like your video?
Maybe you can even engage with the audience
where you either respond to them in those comments
or make polls and reach out to them in other
ways that you have available.
- Is this satisfaction team working on these things?
- We're working on a lot of things.
I would say our work is split in a few buckets.
One is to continue to find these signals
to better understand how users feel.
At the same time, we work with machine learning systems
that use this information to better surface recommendations
that are good for our users.
In parallel, we're thinking of what are others ways
in which we can improve YouTube so
that users are more satisfied?
- What are some of the biggest challenges that the team
has in understanding satisfaction of the viewers?
- Yeah, we have several challenges.
I think, the biggest one is when we either don't have
a good way to understand how a user feels
because it can be pretty frustrating the be asked
for every video like,
"Hey, did you like this?"
"Hey, how, tell us how you felt about this?"
Or "What do you want?"
Finding out the right balance between covering
all users and not becoming annoying to users is,
I would say, the biggest trade-off we're having to make.
- How do you think about, if satisfaction goes up
based on the change but watch time goes down?
Which is better?
- Yeah, that's actually a difficult question to answer
and, I think, a good question.
We actually care about satisfaction a lot.
And I personally (laughs) as somebody working on it,
I'm very happy about it.
But we look at satisfaction as an objective.
So if satisfaction goes up, we are happy with that.
Even if that means that users are watching
a little bit less YouTube overall.
So what we look at is how much time are users spending
watching videos that they're happy about?
And if that time is going up, we don't care
about overall time on the platform going down.
Similarly, we look at not having people be turned off
from recommendation as an important objective on its own.
Even if that means that users are clicking on
a few less videos, we are willing to say,
the user is not being turned off and unhappy
with the recommendation we give, in a strong way
is important to us.
- So do you ever worry about bias in
the people who take surveys?
- So bias is a topic of conversation we
have all of the time.
And what I can say there is we are constantly trying
to both evaluate our data to make sure that
it is representative and not bias, but at the
same time improve it so that it becomes better over time.
- So is there one satisfaction score
for every channel or every video?
- Here, what I can say is that it's not one signal
or one score to rule them all.
Every video has a score for every user and that
is different for each user on each video.
And it might even change over time because maybe
a user is happy about a type of content now
or less happy about it later.
And there are a lot of things that go into that score.
You shouldn't worry specifically about,
"Oh, are the comments specifically important?
"Or is the Like button something that's super important?"
It is combination of all these signals that help
us make predictions for every user and every video.
- Do you have any questions for creators about satisfaction
or anything you'd like to learn from
the audience watching this video?
- On one side, I'd love to hear what are the things creators
would like to learn more about, right?
We are trying to capture audience signals,
how can be better help creators make content
that their viewers are going to be happy and excited about?
On the other hand, I think it would be awesome to hear
from creators on what other ways could we
be measuring user satisfaction?
How can we continue to learn how users feel about
the videos they're watching?
- Great, so if you have ideas for satisfaction,
please leave them in the comments.
If you were satisfied by this video,
what should they do, Eric?
- (laughs) I think, they should like,
subscribe, and comment.
- If you enjoyed this video and wanna see
some more Creator Insider videos about recommendations
and our search and discovery systems, please check out
this playlist right over here.
Otherwise, thanks for watching and keep it real.