- Another big milestone for AI, a competition
between a former debating world champion and a computer.
- So many people are here working on this production.
- Many of the previous demonstrations that we have
seen were the realm of games.
And games have very well defined set of rules.
- You could do simple things like play games,
but you couldn't do complex things
like having a debate.
- There is so many things that are connected
together and so many things need to work smoothly.
I am actually a bit stressed.
I feel the team didn't have enough time to practice.
- So I have to ask you,
who do you thinks going to win tonight?
- Well I'm partial today of course.
What we really want is to create systems that
work with us so that we can make better decisions.
- Ladies and gentlemen, here we go, Project Debater.
- Greetings, Harish.
I suspect you've never debated a machine.
Welcome to the future.
- I started my amateur debating career back in high school.
The teacher for history made the mistake of returning
the exams and saying if you want to appeal
about your score, I'm willing to hear your thoughts.
And obviously nobody seriously considered to do
that except me.
Okay I understand if you think that this is my score.
I have a different view.
Let's argue about that, in writing.
Later on I started to think about writing,
you know comedy sketches.
I started to develop a sitcom called Puzzle.
Oddly enough, the final episode of that sitcom
is entirely focused on competitive debates.
They filmed 16 episodes
and it was a complete failure.
This is how I found myself doing computer science
and artificial intelligence.
- For centuries, people have always saw the games
as a great way to measure your intelligence.
When I play chess, I want to prove to myself I'm
smarter than the other side.
If you take the position that you have to hand code
a rule for every situation,
it is insanely hard.
What we now do is you give them examples,
and they'll learn from those examples
and make their own rules compared to having to craft
all the rules by hand.
- Games have been important for AI but games
differ from debate because the classical game is a set
of equations that's fully understood.
You can draw the rules of chance.
You cannot draw the
rules of conversation, that's impossible.
- Just a couple of days after the Jeopardy! event,
an email was sent to the entire research division
asking us what should be the next era grand challenge.
I was intrigued so I offered my office mate
at the time to spend an hour and brainstorm together.
And towards the end of the hour at some point
I suggested this notion of developing a machine
that will be able to debate humans.
So we submitted the proposal in a single slide.
They told us this is impossible and this is not
a very good plan.
- I think this is a characteristic of a grand challenge
because if it is very clear to you that this is doable,
then probably this is not a grand challenge.
And if it is very clear to you that this is not doable,
then probably you shouldn't start.
It should be somewhere in between but you need to take
into consideration that perhaps you will fail.
- I didn't hear anything for a few months.
Then, February 2012, Hayah Sofail
used this chat service in IBM to text me.
And she said, "Did you hear?"
And I said, "No, what?"
And she said, "It was selected, Debater was selected
as the next grand challenge."
I was very excited and I wrote her back,
"I want to thank you for all the support along the way."
And she said, "Don't thank me yet."
(laughs)
- I'm Harish Natarajan,
and my debating record is that I won the
European Championships in 2012,
the Oxford Intervarsity and the Cambridge Intervarsity.
I've twice won the largest open tournament in the world,
the LSE Open.
And certainly, even if I'm not necessarily the best
debater in the world, I'll certainly be very
high up most people's list.
What I think debating forces you to do is you just
have to defend and justify your own opinions,
but defend and justify often the very opposite
opinions as well.
And I think that process is so valuable because I think
it makes us interrogate the beliefs and the views
which we hold.
- One of the things that happens when you take a grand
challenge is that you have to slice it up
into questions that each of them is less grand.
From the beginning, one thing you would need
is a massive collection of newspaper and journal articles
on the order of 10 billion sentences.
- Then the Debater system needed capability of being
able to automatically identify the exact boundaries
of the claim within the billions of sentences.
- And figure out if those arguments are supporting your
side of the debate or the other side.
- This is something that is very easy for humans to do
based on our knowledge about concepts in the world.
And it is very complicated for machines to do.
Now you need to arrange a meaningful narrative.
- After the opening statement, now there's rebuttal speech.
- This is being able to listen to an opponent
and respond accordingly.
- Those were the main challenges that we had
in front of us.
- And if one of these steps is not working properly,
you will end up with a useless speech.
After we were selected, it was very hard for us
to make significant progress.
Only four months after we started to work,
for the topic physical education should
be mandatory in high school,
the Debater system detects the following
claim in Wikipedia:
"Several studies have shown that lizards display
no benefit from exercise."
This is going to sound ridiculous, okay?
They're going to ask us about physical exercise,
whether this should be mandatory or not,
and how are we going to understand that these claims
are actually referring to lizards and not to
high school students?
It will be very hard to get even close to a meaningful
debate between this system and an expert human debater.
- First time I heard about Project Debater was
one of those small articles probably back in 2014
where they talked about this Debating Watson Project.
And at the time it certainly seemed like a pipe dream.
Humans are rarely just convinced by evidence
and logical rational argument.
It's about really connecting with them in one
of many different ways, some of which
aren't as rational as others.
- What I do is I coach debaters and I tell them
this is what you need to do in order to improve.
And usually a human debater, it just kind of,
their brain understands what I'm saying
and they can convert that into being a better debater.
Sometimes not, but usually.
And in this case, I am saying it to the programmers
and they're saying okay how do we take that learning
process that happens inside people's heads and convert
that into some sort of algorithm?
And that's really where the magic happens.
- It seems to me that this is a motion that basically
for the government it's impossible to defend.
- Well you can agree in general that you'd defend,
like any negative aspects to begin with.
- What I'm saying is that if you will debate me,
not as an expert debater, I will make this point.
Is there an answer to that?
- Being able to show to the world that you can
take a computer program and teach it not just to
play a game that has very specific set of rules
and very specific ways of winning but something
that is so,
so difficult for humans to learn and so subjective in how
you see it and how you hear arguments and how
you respond to arguments.
I think that that's, it's incredible.
When I joined the team, it was not at a level
where it would be competitive.
- My automatic analysis indicates that Yara
suggested that the abolition of the entire
fossil fuel field is going to create and the traumatic
effects of that and lastly,
however considering relevant data suggests some information
that may contradict this claim.
- I first heard about IBM Debater because they were
asking for volunteers to record speeches for them.
Basically, they wanted to build up this database
of speeches from people with a lot of debating
experience and then train the computer
using that database.
- I think at first, Project Debater would sometimes
give arguments that weren't necessarily for its side.
- Such mandatory minimum sentences
lead to overcrowded prisons and rightfully so.
These were the main points I wanted to raise.
- The types of arguments, the connections between arguments,
the way that we sometimes tell jokes
or sometimes we're more serious,
those are all things that they
pulled in for Project Debater.
- I often compare Project Debater to my kids and
their development because I guess my oldest
was born about the same time the program was born.
In terms of personality,
they'd be I guess on par.
My seven year old has a great personality,
they do develop that quite early.
It's scary but also nice to see that Project Debater
is developing a personality as well.
- I don't know whether to call my opponent naive,
old fashioned or just a romantic.
But this way or that, my opponent is wrong.
- I remember the Debater, she called him naive
and it was really hurt.
Suddenly he's laughed at not only by a machine,
but the audience are now bursting out laughing.
My name is Assaf Gavron,
I'm an author most of the days.
I'm in charge of writing what we call generic texts.
Little bits and little phrases that are put together
by the software.
Metaphors or quotes, opening sentences, jokes.
We had a lot of discussions about who is Debater?
- My friend, you are speaking at the extremely
fast rate of 203 words per minute.
Don't hurry, we have plenty of time.
Please slow down and present your arguments calmly.
- We never write the texts to a specific debate.
We don't know what the debate will be.
No, the magic of Debater is that it can argue on
any topic thrown at it under the sun.
So we write a general text that could fit
some topics and then the system has to identify them
and use the right ones in the right time.
- Climate change is what allows them to poison the minds
of young kids.
- Science, we know the earth is warming.
Will there be more droughts?
Yes.
- How definitive is the evidence?
Is there any room for debate?
- This is the Natasha Hall Show, on CJAD800.
- We are speaking with Dr. Aharanoff,
she is the global manager for IBM's
artificial intelligence Project Debater,
thanks for being here.
How nuts is that it could take artificial intelligence
to teach us to understand other human perspectives better?
- I think that's where AI should be going.
- Why?
Why do we need robots to teach us how to debate?
- Ultimately making a decision involves how you weigh
your different values.
It's not all about information and I think
that AI is not gonna do for you.
But it can give you all the information so you
can make a better decision.
You caught me in one of the most tense days ever.
I'm flying in three days.
Obviously we had a bug that we found last week, naturally.
You see results that are just not consistent.
You say I'm gonna run this tomorrow,
I'm gonna get a different result,
and you have no idea where it came from.
It was really what's called a ghost bug,
it just didn't make any sense at all.
This was testing how much we can stretch our nerves.
So that was fixed by a patch to Java and then we had
to rerun everything and go from there.
But it's fixed.
- Every time I try to select it.
- Test again.
- We're gonna just play something constant and I'll
bring it in slowly.
- It's a single debate.
It's just one debate, so in a single debate,
many things can happen.
And Harish is a much better debater than any debater
we faced in the past.
- Let me introduce John Donvan.
(crowd applauds)
- I host a debate program, Intelligence Squared US.
Our goal has always been to raise the level
of public discourse.
This truly is a first for us.
The first time that an artificial intelligence,
namely Project Debater will be on our stage
arguing with a human being.
And may the best debater win.
So let's get started, let's first meet and applaud
our debaters first, arguing for the resolution tonight
will be IBM Project Debater.
Representing the rest of us,
please come to the stage, Harish.
So, to reveal now the resolution of the evening is this.
We should subsidize pre-school.
We are going to ask you to vote before and after
the arguments using your mobile phone to tell us where
you stand on this position and to tell us whether you were
persuaded or not persuaded by one side or the other.
Project Debater actually has a gender.
She will be arguing--
(crowd applauds)
She will be arguing for the resolution,
ladies and gentlemen, here we go, Project Debater.
- Greetings, Harish.
I have heard you hold the world record in debate
competition wins against humans.
But I suspect you've never debated a machine.
Welcome to the future.
I will argue that we should subsidize pre-schools.
For decades, research has demonstrated that
high quality pre-school is one of the best investments
of public dollars, resulting in children who fare
better on tests and have more successful lives
than those without the same access.
In December 2015, researchers at Duke University
found that students who enroll in pre-school are
39% less likely to be placed in special education programs.
A statistical summary of studies by the
National Institute for Early Education Research found that
high quality pre-school can create long-term
academic and social benefits for individuals and society,
far exceeding costs.
While I cannot experience poverty directly,
and have no complaints concerning
my own standards of living,
I still have the following to share.
The Centers for Disease Control and Prevention
reports that universal full day pre-school creates
significant economic savings in healthcare,
as well as decreased crime, welfare dependence,
and child abuse.
Why don't we examine the evidence and the data
and decide accordingly?
Thank you for listening.
(crowd applauds)
- Well thank you very much, everybody.
It's a pleasure to be here for this historic event.
There was a lot of information in that speech,
and lots of facts and lots of figures.
The problem though is the reality of subsidizing pre-schools
is one which does not deal with the underlying
problems in society.
I'm not sure if you massively increase the number of people
going to pre-school they're all going to be the ones
going to the high quality pre-schools.
- Mr. Natarajan suggested that pre-schools should not
be subsidized because this will reduce their quality.
If anything, the opposite is true.
One of many reasons is that subsidizing
attracts more skilled and qualified people to the field,
improving the quality of pre-schools for all.
Pre-school is pushing that child to learn in a competitive
environment at the age of three or four,
when you realize you aren't necessarily as talented
as someone else.
That creates psychological damage for many children may
mean that pre-school is actively harmful.
- My opponent claimed that pre-schools are harmful.
I believe my argument suggested that the benefits
outweigh the potential disadvantages.
- You cannot fund everything.
I think this is simply empirically true and you
have to make choices and you have to make trade-offs.
Many middle class parents and many people from
upper incomes already send their children to pre-school.
You are taking money from all taxpayers to help those
individuals who are already often the best off.
To be clear, my intention is not to leave a suitcase
full of money for everyone to grab at will.
We are talking about a limited,
targeted and helpful mechanism.
- I don't think that Project Debater has helped those
individuals she identifies as the most important,
but in reality has hurt them.
- Thank you and that concludes round three
and the argument phase of this debate.
(crowd applauds)
We're on our way to making history here.
We would like to ask you now to complete that process
by using your phones and choose your position
where you stand now that you've heard the arguments
from both sides.
At Intelligence Squared, we deliver victory
to the side whose numbers have moved up the most
in percentage point terms.
Harish Natarajan, his first vote was 13%,
his second vote was 30%.
He pulled up 17 percentage points.
That is it, Harish Natarajan arguing against the
resolution subsidized pre-school declared our winner,
our congratulations to them.
Oh, and we have the second vote.
Which of the two debaters better enriched your knowledge?
Let's see what that number is.
Project Debater better enriched the knowledge
of the audience on that side, so,
a little bit of a split decision.
(crowd applauds)
This is amazing.
And I think regardless, we made history tonight because
Project Debater held her own.
- It's fine to disagree.
It is not fine not to listen
and not to really try to understand the other side.
We see artificial intelligence technologies everywhere.
We use that in our cars,
and we use that to select music,
and we use that when we try to buy something.
What about discussions between us?
- Thanks for joining here in Geneva for the
AI for Good Global Summit 2019.
- I think the Debater is really just the beginning.
It touches on a very fundamental problem,
understanding the pros and cons for taking better decisions.
So for example a government which is considering a policy
and the citizens that are going to be affected
by this policy.
I don't think it will replace the decision makers
but it will help them.
- (speaks in foreign language)
- Greetings all, the citizens of Lugano have contributed
more than 1,000 arguments in support of the claim
that we should further explore the development
of autonomous vehicles.
The following is a summary of their thoughts.
The crowd raised four issues.
The first we'll demonstrate how autonomous
cars reduce accidents.
Another point is that autonomous cars
can help elderly people.
Then the topics of autonomy--