Today on "Hello World,"
we are sticking close to home for once.
We're gonna travel virtually down the road
to Stanford University,
where there's a research lab
that is doing some really cutting-edge work
around AI technology, video analysis,
and video manipulation.
A group of researchers there have done some projects
in things ranging from creating fake tennis players,
versions of Roger Federer and Serena Williams
that can play matches against each other.
And they're also doing this crazy analysis of cable news,
where they've had an AI watch 10,000-something hours
of cable TV news and find out which topics dominate,
who's getting the most airtime between people
like Trump and Biden.
And so, it's just, it's these fascinating applications
of the cutting edge of where AI and video meet.
Some of it, I think, will blow your mind a bit
and raise some difficult questions.
And so, with that, let's head off to Stanford University.
Most generally I'm a professor of computer graphics.
And so, my students and myself and colleagues
like creating new interactive experiences
that were never possible before.
That's sort of our mission, what makes us tick.
I was on Twitter one day and I'm just scrolling through.
And I'm a tennis nerd,
and so, I caught a video of this thing
of Roger Federer playing against himself at Wimbledon.
And that obviously caught my attention
since that's kind of impossible.
And, you know, I started digging into it,
and I saw this was based on some video analysis
and AI technology that you guys have been working on.
A lot of us in the world right now
are really interested in different forms
of generating, analyzing, or manipulating video.
And so, I actually think it was an idea from my colleague.
He said, "Hey, you should take
"all this broadcast sports video that's around
"and make a really cool video game out of it."
Yeah, and just to break it down,
and you tell me if I'm understanding this right,
but you feed your systems all this video,
just the raw video of these matches being played.
Each player that you're focused on,
you're learning their, the style of play,
what shots they're likely to hit.
We took two ideas that are out there in the real world.
We took sports analytics,
which all the leagues are doing all the time,
and that they're analyzing the video.
And then we took a very basic idea
from computer graphics circa 2000,
which is if you want a character to do something,
record them doing many things,
and then mix and match
a bunch of little video clips together.
And we did use a number of modern deep-learning techniques
to really fill in some of the gaps.
I mean, obviously, deep fakes are a big concern
with modern AI.
Are there, you know, legit, I don't know,
concerns that could arise from something like this?
I think there are absolutely always legitimate concerns,
but a lot of what we did was based on technology
that existed 15, 20 years ago.
And I agree that there's an inflection point.
When it starts looking real, very realistic,
there's great power that comes with that.
But it is not a new thing.
For a while, you're like,
oh wow, this is Roger Federer hitting the ball.
And then every now and then, you're like,
Oh, it moves-
Absolutely, yeah.
A little weird.
I don't know if glitchy is the right word.
But then at the same time, I was like, you know,
I play tennis video games,
and there was a realism to it that you don't see
in a lot of those games.
You could see that this is, maybe,
the future of where this stuff is going,
is like I can actually play a simulated Roger Federer.
I mean, absolutely.
The next step going forward is,
is working with folks that can give us access
to a lot more video,
'cause we did what we showed with only
two or three matches from both of these players.
If you gave us a much bigger database
and the next set of techniques,
I believe we can generate
a very compelling visual experience.
You guys do a variety of different projects over there,
and you also have this project where you've done
this really broad analysis of cable news,
looking at both what is said by analyzing transcripts
and also who is saying it.
Who's on TV the most between men and women,
different pundits, you know,
different people in the news
like Trump and Biden, obviously.
We're taking an extremely large corpus of video.
We have almost 24/7 broadcasts
of CNN, MSNBC, and Fox News since 2010.
So, we have transcripts and we have video for all of those.
it's provided by the Internet Archive.
And what we've done is we've basically turned it
into an enormous library.
We've indexed it and we allow the public
to search those transcripts,
as well as who is on screen.
And so, we're really excited to see how,
if we give the public, whether that be media watchdogs,
whether that be journalists,
whether it be scholars or hobbyists,
the ability to essentially search this library
of who and what was on the news,
we think that some very positive things might happen
in terms of understanding what gets presented,
who gets the opportunity to present it,
as well as what biases or
dispositions the various channels have.
And so, I noticed,
I mean, obviously,
historically there are watchdog groups
that look at this sort of thing.
But it's all, it's a manual process, right?
Some poor human has to watch endless hours
of horrendous cable news.
That's correct.
And the big difference here is that we have taken
some very manual, painstaking labor
by folks that wanna understand what's being communicated
and how to computer-automate a lot of that labor.
We are just a few blocks-
One of the realizations that came out
was an imbalance between male and female hosts.
There's a lot more male hosts.
One thing I was surprised about was,
I mean, Fox News has almost a one-to-one ratio
between male hosts and female hosts.
They had by far the best balance on the host side,
but on the pundit side,
they were, I think, the worst.
That's correct.
And one of the real,
the things that we're interested in doing
is that we all have a narrative
about what's going on in the world.
And this is about putting data behind that narrative.
And the conclusions, sometimes, are quite interesting.
I wanted to talk just a little bit
about the nuts and bolts of how you guys put this together,
'cause one thing I did not realize,
Amazon has this facial recognition tool
that you can buy as a service.
What we are surfacing to the public are the results
of Amazon's celebrity recognition service API.
I'm excited to introduce for you a new service
called Amazon Rekognition Video,
which does real time and batch video analytics.
We'll detect objects and faces and scenes.
It is a form of doing facial recognition at scale,
which has been talked about and continues to be talked about
at length in the media right now,
because it is an extremely controversial technology.
Amazon can scan your face without your consent
and sell it to the government
all without our knowledge, correct?
Yes.
Now, on one hand, that's what we are doing.
We are running Amazon's face recognition software
on every frame of the news for the last decade.
But we think that this is an application of face recognition
where the potential for harm is low.
It is run only on individuals
that have appeared on cable TV news.
But, well, we thought that the ability
to audit what is going on on the news
was a positive use case,
but we had to make a judgment.
And the judgment we made was that the benefits
outweighed the potential harms.
One thing I think is worth bringing up,
which you guys address,
the facial recognition.
It does way better on kinda like white guys
than it does on females,
and it also has some struggles on gender,
especially for people who might identify in different ways.
Is that right?
I mean, it's basically strongest
on white male pundit than anything else.
Most of these services have shown scientifically
that there is bias in these systems,
and so any conclusions that you take
from automated machine learning analysis
need to be inspected very carefully
to determine if that bias is playing a factor
in the results.
Our goal is to create interesting new capabilities,
tools for analysts or creators,
tools for artists, if it's in entertainment.
And so, like any tool builder,
you wanna use the best tool for the job.
And when AI provides new opportunities
and is the right tool for the job,
by all means, we should use it.
But we also look at it with some skepticism
in that if we go back to the tennis project
that we talked about earlier,
shoot, most of the reason why that looks so good
is because we took human knowledge of how tennis works
and encoded it in the computer,
and we used the AI to fill in the gaps.
But the main knowledge is still human,
and I actually think when people stop doing that
and rely completely on data
or completely on AI to build systems,
that's where stuff starts going wrong.
You know, for better or for worse,
computing has gotten very, very powerful in the last decade.
And there are amazing things that we can do with that power,
and there are significant concerns and significant issues
that that power is bringing to bear on all society.
There's a lot of responsibility to think very thoughtfully
about what we are doing and why.