I would say it's easily in a million lines.
A million lines of code
Absolutely.
The algorithm is meant to indicate all of the different methods
We use who decide which search results we'll going to show you
and in what order for which recommendations we'll going to choose for you
I think people when they refer to the algorithm they mean all the code
that we've written to do those things
I don't think of it as one thing, I think of it as lots of little pieces that are
trying to accomplish the overall goal of connecting you with what you're looking for
or what we think you might want to watch
it's not like a piece of code hidden in a safe or something
(Laughter)
there's no piece of code hidden in a safe
there's a lot of code and it scattered all over the place in our code base
if you look closely at it it will resemble much more accounting than any
sort of dark sophisticated thing.
there is whether it's the algorithm or
whatever is there is a degree of secrecy is in there for good or bad reasons
I mean you you can't be completely open about all of this, can you? or you're not
open about all of it seems
is that a fair comment?
I mean I think we're quite open about what we're trying to achieve
and I think we're open about the instances where we know it's not
working and we think it could be better
I tend to be a little bit more reticent to discuss exactly how we're doing it
and there's a few reasons first of all we don't want people to try to game
what we're doing and sadly we do see that on YouTube and so everything do we
have to think about the opportunities for gaming it I mean we're using certain
kinds of information to help people find the videos they're looking for
That depends on on that being that information coming from sincere viewers
who were actually looking for things
now if instead it came from a robot
it wasn't really looking for anything it was just trying to make sure that a
particular video showed up at the top of some search results are you know how to
tremendous number
views then then we can't use that anymore and so it's not just that
there's a few more instances of poor search results or inflated view counts
but but it's worse than that because actually that's what that means that
that we can't use the thing that we were using anymore and so now we have to
you know, we have to take that away as a possibility for helping associate
the viewers with the videos they want to watch and then move on to the next thing
and then of course that thing may also be abused and so, you know, we have to
try to protect these these signals as we call them so that so that we don't lose them
and you know that and then we're left with very little to actually go on
I don't want to lock us into a particular way of doing things
you know say for instance that five years ago
ranking the search results by the view count may have been a great thing to do
and that was because at that time
view count was one of the best signals
that you could find for what are the quality of the video
well since then we've found other things that are even better
and we don't want people to become upset when they say hey wait a minute these videos
are ranked in the order of the view count
that's wrong what's not wrong because we like I said we would have done an
experiment to find that there was actually a better way to do it yes we
take the view count into account but there's some other thing that we might
want to do for instance a very good example is if it were just ranked by
view count you've never seen any new videos
so you type plane crash after a plane crashes and you'd see the oldest plane
crash footage we have if we want to have some video of the new incident
we can't rank everything by view count we don't want to commit ourselves
in a way two people thinking that this is the way it has to be and that we can't make
any changes because where we have to we have to make changes
you mentioned view
I mean view account is very transparent signal because of the design of YouTube
can you mention some of the other signals you're using
sure what one of those that we started using
a couple of years ago now I guess what was watch time. So what we noticed was
that was that when people looked for videos
it was often the case that they would see one that looked like the video they
wanted to watch but then once they got on that watch page it wasn't actually
the video they wanted to watch
maybe it had a thumbnail that was you know very misleading or title that was
particularly misleading
now fortunately in most of those cases eventually they got to the one they were
looking for and often that was because that would be somewhere on the related
videos but in looking that situation we thought when 'Gee that's kind of crummy'
because why should I have to click two or three times in order to finally get
to a video that I was looking for
we tried to figure out how to fix that went well people tend to know fairly
quickly whether or not the video they've landed on is the one they're looking for
the best example I remember of this was we saw one where there were several
videos that had titles about a particular boxing match and one of them
especially had a thumbnail of a guy like he was just about to be hit and miss you
know its fist was right up in the other guy's face and everybody was going to
click on that one
what turned out that the video behind that was a person talking about that
fight and it had no footage of the fight and eventually the viewers would finally
get to the one that had some footage of the fight when we looked into that
situation we realize that one way to detect the difference between these was
one of these videos wasn't getting watch very much and the other one was once
very loan or watch for very long exactly one of them wasn't getting watch very
long and the other one was and so instead of just ranking the videos by
how often they were clicked in response to a query we rank them for how much
they they were watched in response to a query and so that helped that situation
quite a bit
how often are you twiddling the dials and refining this and tweaking - is
like a daily thing with little things that can change or is it like a
once-a-year big implementation
we've we've always got at least I don't know ten different changes that were
working on and they may take somewhere between a few weeks to several months to
finally launch and there's usually some some analysis and intuition and then
there's some implementations some changes that you're going to make to the
code and then there's lots of experimentation and evaluation in order
to confirm that this change actually benefits the viewers and so how many
times do we do in a year maybe 50 times a year or a hundred times a year
other things that believes that can be pulled like okay today we're going to
put twenty one percent emphasis on watch time and no no it needs to be nineteen
percent is it is it like that or not
um it's not really like that because all of those settings or levers or or all
the code that we've written to make those kinds of decisions has been has
been tuned based on evaluation and analysis of some kinds of experiments or
other ways we have of rating changes and so it would make sense for us to walk in
and saying like you know I'm feeling like we ought to put a higher weight on
something even if i were to feel that we ought to put a higher weight and one
watch time for instance what that would amount to is some of the engineers doing
several weeks worth of experiments to see whether or not putting a higher
weight on watch time resulted in more people getting to the videos they
watched faster
we can see
the data that we've got about now the data that we get back is completely
random it depends where when the server crew received that message to put in same
computer's memory
so this time we're on it is actually giving us some use for the information
because when we have demonstrated some evidence you know in the broader subject
another way that we might find to recommend things too