- Something that's striking about Chatbots
is the number of them.
Intelligent agents and virtual companions,
there aren't very many of them
because they're quite complex to build.
Chatbots, because they have a much more narrow focus,
people can think of applications for them much more easily
and the development platforms for them
are really quite accessible.
So we've seen a huge Chatbot rise.
In 2015 we saw Facebook M,
that was one of the first sort of well known Chatbots,
and then development platforms coming from Facebook,
Microsoft, LINE, IBM, and so on.
It really sort of spurred on this development
to the point where we were having
hundreds of thousands of Chatbots
and companies like Slack were even introducing
development funds.
And analysts like Gartner were making some pretty strong
statements, saying in the next few years,
we'll see 50% of enterprises spending more on bots
than they do on apps.
So there's a real surge here
but if you move forward a couple of years
to sort of where we are now,
you can see some challenges starting to emerge
and even last year, and this is what started
some of the conversation for Johnathan and I
at CHI last year, was you could see some people
beginning to raise some questions
around bots are in the marketplace
but they're not really meeting the expectations
and there was some disappointment starting to creep in.
Although, the trend is still positive
and they continue to be really useful
and the growth continues, but you do see things
like Facebook M shut down and then,
people asking questions like David Feldman,
who was one of the principle developers
for Facebook M, asking questions
about what happened to Chatbots.
So Johnathon and I were thinking about this
and we have a theory for this, which is this is probably
the Gartner Hype Cycle taking place.
In the beginning there's a lot of excitement,
people rush in, a lot of bots are being developed,
but then there's this point
where either you reach saturation
or some expectations are not being met so we start to dip.
And so we're in what Gartner would call
this Trough of Disillusionment, but the really encouraging
thing about this that we feel is
it can work its way back up again
and it can work up to what Gartner would call
the Plateau of Productivity.
So then the question becomes how do we move forward
along the Slope of Enlightenment
and into the Plateau of Productivity?
So this is where Jonathon and I were sort of reflecting
on what are the things that we've learned
from looking at the literature and from our own experiences
with developing these Chatbots and working on agents.
So here are some things that we think,
there are more than this, but here are some things
that we think really lead to success
when developing Chatbots.
So keeping a really narrow focus really helps
and finding areas that have specific markets.
So again, thinking about things like a travel agent
and being able to help people with travel recommendations
or maybe even kinda book a flight.
When things are really specific like that,
they're really kind of helpful.
The other thing that was quite striking,
and this is where we get into the conversation
about Humbots, was we realized that many times
bots are actually continuing to reach out to humans too
and that was kind of an interesting aha moment
because I think a lot of the industry
is trying to enable bots and agents to go it alone,
to just be the tech on itself
and maybe the tech's not really ready
for some of the things we want it to do.
So there's nothing wrong with having a human in the loop
and embracing this idea of Humbots.
So just to talk about this one a little bit further,
there was a paper a couple of years ago
around a bot called Calender.help
and this was a bot that was to help people book meetings.
It's a pain point that a lot of us have,
it seems like it's a really straightforward task.
You know where the rooms are, you know the people,
you know the time and so on, but it turned out
to be surprisingly difficult
and this is what Cranshaw and folks were talking about
a couple of years ago in this CHI paper.
And they actually found that only 5%
of the successful meetings booked could be done
by the tech alone and that humans had to be in the loop
the rest of the time and they ended up developing
and escalation path which had different levels
of human involvement.
But this is a good example of how humans and technology
can continue to work together
and deal with some of the challenges we're starting
to experience in the era.
So in terms of further challenges,
like Calendar.help was showing us,
even the simplest task can often be quite complex.
You can easily get into a situation where
if you don't maintain boundaries on what the task is,
you get mission creep and either the user is not sure
about what the bot can do or the bot itself
is starting to creep into other areas.
The travel bot, for instance, could start finding activities
that are unrelated to travel.
I think we talk about this one a lot in the field
but conversation and natural language
is surprisingly difficult.
As humans, we've been doing this for a really long time
and so we're experts so the bar is really really high.
Along with that come questions
around trust and transparency, personality
and especially during the first run experience.
It's really critical that people have a good sense
of where information is going,
if necessary how to retrieve information,
they just feel that the experience is trusted.
Especially when it's easy to make Chatbots
on development platforms
you don't know who's making these things
unless you communicate that information.
And then the last challenge that we wanted to call out
was one we call the demo trap.
So this goes back to, we've made some pretty good platforms
to develop bots and it's fairly straightforward
to make them but to be successful
it's not just about making it and getting it
into the marketplace, it's about working afterwards
to see how the product is being used,
looking at the data that's being worked on
and really continuing to improve.
So what does all this mean for the future?
Well, like we started at the beginning,
oftentimes it's best to look back.
So we can look back to some work that Licklider did
in the 1960s again and he created these three stages
of human-computer interaction.
The first phase was beginning in the 1960s
and has continued forward to about where we are right now.
And the second phase is moving into this area
of around human-computer partnerships.
It's probably the phase that we're in right now.
And it's hard to define how that's gonna go
and how long it's gonna last
but Licklider said, "It could last 10 to 500 years
"And it should be intellectually the most creative
"And enticing part in history of humanity."
That's a long period of time but it's kind of encouraging
'cause it'll also mean that conferences like this
probably will keep going for 500 years
as we work it all out.
And then phase three is what Licklider calls
the ultra-intelligent machine,
which is the period where we'll get to
the full general artificial intelligence.
I'm gonna stop there, thank you.
(audience claps)
- [Host] If you have a question
please come to the microphone and introduce yourself.
- [Ben] Ben Shnider, University of Maryland,
thank you, interesting review, did you in your discussions
or taxonomy look for the places where instead of the agent
doing it in collaboration that the person did it themselves?
I mean, I see that for a lot of tasks
the different kind of interfaces enable them
to do it themselves, like making flight reservations
or a lot of different tasks.
So under what circumstances is it better
for a person to do it themselves
and when is it better for a conversational agent
to help them?
- Yeah I can't say that we have a direct answer
for doing that, I think it's a really good place
to investigate next.
The thing that we were generally finding
goes back to how complex tasks are
and there were very many places where it was good
to have human involvement or at least some level
of curation by the human so there ended up being
a lot of places where humans needed to be involved.
So I think as we take this forward,
the human and bot partnership would be really--
- [Ben] I'm not asking that, let's say where the human user
does it themselves not as another unit helping
but they do it themselves individually.
- Oh I see, no we should look at that further.
- [Claude] Hi I'm Claude Guinness from IBM Research.
When we looked in our research into these issues
we also see what we think there's another dimension
which is the speech versus Chatbot text
and how that sort of make a map into those
and I wonder if you have look into this issue?
- Can you say more about what you mean by speech--
- [Claude] Like Alexa using speech, speech-based Chatbot
versus chat, text-based Chatbots.
Like the difference, maybe give an example
when most of intelligent systems are going the direction
of speech while most of the Chatbots today
are in the text world
and we wonder why it is going there
and what happens in the two empty dimensions of this.
I'm not sure if I'm being clear.
- I think so.
I think we're in a transition phase right now
where intelligent assistants are moving much more
towards an audio-speech based way.
I could see Chatbots starting to go more in that direction
as we move forward too, so it's hard to say
what the reasons are for that
but I can see why you're saying that.
- [Claude] Okay.
- [Mark] Hi, Mark Schillinger from Google
there seems to be an assumption in here
that Chatbots are going to be talking to humans
and humans to Chatbots, did you think about a future
in which my Chatbot talks to your Chatbot
and I'm out of the loop?
- Yes, we didn't cover that in here
but we have been thinking about this a lot
in terms of what are all the relationships
that could happen between one Chatbot to another Chatbot,
a Chatbot to an intelligent assistant,
and then also what it means to have a human
involved in that too.
I think that's gonna be one of the next phases
that we as a discipline start to investigate further
because it gets complex really quickly,
trying to figure out what those combinations are
but also what the relationships are as well.
Thank you for bringing that one up.
- [Host] Let's thank Richard again for the great talk.
(audience claps)