LAURENCE MORONEY: Today, I'm meeting
with Paige from Google Search to talk about the coolness that
is Now on Tap.
So, Paige, you're a Product Manager
for Google Now on Tap, right?
PAIGE DUNN-RANKIN: Right.
LAURENCE MORONEY: So, tell me all about Google Now on Tap.
What is it?
PAIGE DUNN-RANKIN: It's a new feature for Android
Marshmallow, and it helps bring me related information
and actions.
Whatever I'm doing on my phone, I can just use Now on Tap
to get information and actions.
LAURENCE MORONEY: Ooh, are you going to show me?
PAIGE DUNN-RANKIN: So, here is a chat with my friend Brian.
So, he says, hey, thanks for picking me up at the airport,
gives me his flight number, and then
I'm asking him about dinner.
So, what I would typically do in this situation
is copy and paste some of these things.
LAURENCE MORONEY: Yeah.
PAIGE DUNN-RANKIN: You look up the flights number--
LAURENCE MORONEY: It's awkward to copy with your fingers.
PAIGE DUNN-RANKIN: --and you might check out
a couple different apps to see about the restaurant he
suggested.
So, with Now on Tap, it makes all of that super easy.
You tap and hold on the Home button right there.
LAURENCE MORONEY: OK.
PAIGE DUNN-RANKIN: And then you get all this information
and actions--
LAURENCE MORONEY: Oh, nice.
PAIGE DUNN-RANKIN: --related to what you are doing.
So, here is the restaurant that he suggested.
Here is the flight.
LAURENCE MORONEY: There's his flight.
Oh, and he's delayed.
PAIGE DUNN-RANKIN: I can see it's delayed, so that's useful.
I can create a calendar event for this dinner
that we're having.
And it actually used natural language
to predict this calendar event, which I think is pretty cool.
Because we had mentioned dinner, tonight, and the place
all in different parts of the conversation,
and it combined all of those into one action.
And so it's trying to predict what I want to do
and also what information I need.
LAURENCE MORONEY: OK, cool.
Now, beyond just the information in a chat like this, can it
also, when you're in a particular location,
it can predict the kind of things
that you do based on time?
Is that right?
PAIGE DUNN-RANKIN: So, that's more of like a Google Now use
case.
So, Now on Tap is actually building
on top of what Google Now has already done.
So, Google Now predicts what's going on in the real world
and what information you need that's happening around you.
Now on Tap is about what's happening on your phone.
LAURENCE MORONEY: Oh, OK.
Cool.
PAIGE DUNN-RANKIN: And so, when we were thinking about this,
this was one of these crazy ideas we had.
What if you could get Now cards for what's
on your screen at any moment, any screen that you're on?
LAURENCE MORONEY: When you were working on this kind of thing,
it must have been really challenging and exciting
to think about the context of what people do on their phones.
How was that?
PAIGE DUNN-RANKIN: Yeah.
It was a completely new idea.
It was something that's never really been done before.
And so we looked at a ton of screens.
We were all donating our screens as data.
And we had people going through their top apps
and making sure Now on Tap worked well.
And yeah, it was a totally new project
that started out as just a small team doing some prototypes
and then grew into this product that we launched in less than a
year.
LAURENCE MORONEY: Right.
And it's in Android Marshmallow, right?
PAIGE DUNN-RANKIN: Right.
LAURENCE MORONEY: So, cool!
So, if I'm an app developer and I
want to take advantage of something like this, what do
I need to do?
PAIGE DUNN-RANKIN: So, the great thing about Now on Tap
is that developers don't really have to do anything.
And so it just works automatically on most apps.
There are a couple of things that we really
recommend you do.
So, one of those is getting your app into App Indexing.
Basically, if you have information
that should appear in search-- so, for example, you
have Yelp with a bunch of Yelp reviews,
or IMDB with a bunch of pages about actors--
we actually show those apps in search
if you've gone through and participated in App Indexing.
And that's where we know exactly where to go within Now on Tap.
So, if you noticed, we have all those app icons on the cards.
And then when you click on those,
it goes to exactly the right place.
LAURENCE MORONEY: OK, wow.
PAIGE DUNN-RANKIN: And the way that we do that is we actually
know from search where to point to.
One of the other issues that we have
is when people do crazy things with views.
And so that's a little more rare, but as long
as your app is basically working well with accessibility, APIs,
and such like that, it should work well with Now on Tap.
LAURENCE MORONEY: What kind of stuff
did you guys encounter while trying to build something
into Android?
PAIGE DUNN-RANKIN: One of the things
that was really exciting for me was seeing
how the API was designed.
So, Now on Top is actually built on top of an open API.
So it's an API that anyone can build
an assistant that takes advantage
of this information about the view underlying on the screen
and also this layer that comes up on top of the page.
And so that was really cool to actually design that API
and work with the Android team.
And then we also had some pretty funny challenges.
It was actually pretty hard to tell from context
what you should be showing.
I guess it sounds hard, right?
LAURENCE MORONEY: Sure.
PAIGE DUNN-RANKIN: And it was hard.
LAURENCE MORONEY: I'm sure some things went wrong
along the line.
PAIGE DUNN-RANKIN: Yeah.
We had funny-- you know how there's a band called Yes?
LAURENCE MORONEY: Yes.
PAIGE DUNN-RANKIN: We would trigger for things like that.
And it was a little bit dumb at the beginning.
But then we started seeing these things
and then fixing them, sort of tuning
to the context around the words.
LAURENCE MORONEY: So, what, if I was in a messenger
chat with a friend and the word "yes" was in there,
and I did Now on Tap--
PAIGE DUNN-RANKIN: Yeah, you might get a card for the band.
LAURENCE MORONEY: --then I'd hear all about Jon Anderson.
PAIGE DUNN-RANKIN: Right.
LAURENCE MORONEY: Oh, I love it.
I'm sure lots of other things could go interestingly wrong
with that.
PAIGE DUNN-RANKIN: Yeah.
One of the things that we have actually struggled with
is making sure to always find something
useful on every screen.
Because, if you can imagine, there's
just not entities perfectly spelled out on every screen.
So, one thing that we introduced that I think is really cool
is basically ways to find trending searches
on your screen.
We actually know a lot of people are searching for right now.
And so we can actually incorporate that into Now
on Tab.
LAURENCE MORONEY: That's pretty cool.
And, obviously, we're just talking about English right
now.
So, it's also available in other languages, right?
PAIGE DUNN-RANKIN: Yeah.
At this point, we're launched in I
think eight or more languages.
LAURENCE MORONEY: Nice.
PAIGE DUNN-RANKIN: And it was a challenge, you can imagine.
This all relies on natural language processing,
and so you really have to know all the nuances and the quirks
of the different languages in order
to not make those types of "Yes" examples,
those bad-- "overtriggering" is what we call it.
And so--
LAURENCE MORONEY: "Overtriggering."
I'll have to remember that one.
Whenever I try to speak French and insult somebody,
I'll just say "I'm overtriggering!"
[LAUGHTER]
PAIGE DUNN-RANKIN: Yeah.
So, luckily, we had a team in Japan dedicated to this.
We also had a lot of engineers in Europe.
That's where the core Now on Tap team is based.
And so they all speak like seven languages.
LAURENCE MORONEY: Wow.
Nice.
PAIGE DUNN-RANKIN: Yeah.
They were able to help out.
LAURENCE MORONEY: Nice.
And it works just as well.
PAIGE DUNN-RANKIN: You know, Now on Tap
is actually really useful.
Some engineers in Japan have been telling me
it's really useful for them, because it's
hard to type out all the characters.
Copy and paste is even more useful in Japanese, basically.
LAURENCE MORONEY: That's true, yeah.
Of course.
PAIGE DUNN-RANKIN: And so Now on Tap is incredibly helpful.
LAURENCE MORONEY: Oh, nice.
So, back to developers.
You did mention App Indexing.
And App Indexing is part of it.
But I hear you've got some great resources for developers
for everything that they need to get into Now on Tap.
PAIGE DUNN-RANKIN: Yeah.
If you're really interested in making your app work perfectly
with Now on Tap and learning about these assistant APIs,
I suggest taking a look at our developer docs on this.
We have a great article that's all about the Assist APIs.
LAURENCE MORONEY: One article's all they need.
PAIGE DUNN-RANKIN: Yeah, it's really pretty simple.
LAURENCE MORONEY: Nice.
Well, thank you, Paige, and thanks for coming today.
PAIGE DUNN-RANKIN: Thank you so much.
This was great.
LAURENCE MORONEY: If you've got any questions for me,
or if you have any questions for Paige,
just please leave them in the comments below.
And don't forget to subscribe to the Google Developers channel
on YouTube.
Thank you.