[MUSIC PLAYING]
MARY CHEN: Hey, everyone.
You heard a lot of new announcements
and learned a lot throughout the day.
So we're just going to end the event with a very
casual fireside chat.
We'll be going over some of the highlights
and answering commonly asked developer questions.
My name is Mary.
And I'm part of the Google Assistant Developer Relations
team.
And co-host of this fireside chat
is my colleague and previous office mate, Jessica.
JESSICA EARLEY-CHA: Oh, remember when
we used to go into the office?
So weird.
But yes, I'm Jessica.
And I'm a developer advocate.
And we've collected questions from developers.
And we wrote them down on Post-It notes.
And we've actually recorded this a few weeks back.
But if you have any questions during the fireside chat today,
please ask them in the chat window and we'll
answer them live.
So grab a drink and join us for some afternoon tea.
I'm actually drinking Jasmine tea right now.
How about you, Mary?
MARY CHEN: I'm having a chai latte.
JESSICA EARLEY-CHA: Mm, oh my gosh, yum.
And also, look out for any special cameos
from our fur babies throughout this conversation.
MARY CHEN: Cool.
So before we begin the Q&A, here's
a quick recap of the main highlights from today.
If you're a mobile developer, App Actions
is now available for anyone with an Android app.
You can now use App Actions to integrate
your app with Google Assistant for so many different use
cases.
And if you're a native developer,
we want to just remind you that we hear you.
We're heavily investing in discovery and monetization
opportunities for you.
And we're constantly improving the developer experience.
So with that, let's jump into App Action questions.
So Jessica, I get this question a lot
previously for App Actions.
And I think now that it's readily
available for all developers, let's tackle it first.
As an Android developer, how do I get started?
Do I need to build something new,
or can I just innervate my existing app?
JESSICA EARLEY-CHA: Yeah, that's a great question.
And usually the first one because it's
the most immediate.
So the great thing is, with your Android app,
what happens is you already have all that functionality.
And adding App Actions adds that voice capabilities
to it, which is great.
So all you need to do is support deep links.
Once you've done that, you can build out your app action.
We've broken this down into four steps.
And the first one is to tell the Assistant
on how to use your app.
And so this is giving the Assistant just,
hey, I do x, y, z.
And then the second step is, when
the Assistant does have those requests,
how do you process that.
These are your fulfillment URLs.
And then the third step, of course,
is testing it out and making sure it works.
And then the fourth is deploying it.
And if you want a hands on tutorial guide
on how to build this, you could go and check out
our codelab that's all about integrating an Android
app in App Actions.
So with that, when it comes to telling the Assistant on how
to build or what you actually can do,
those are called built-in intents.
And we've recently launched many more built-in intents.
And I believe we now have over 60 built-in intents.
And I can't list them all.
So Mary, I have a pop quiz for you.
What are some of the built-in intents that are available?
MARY CHEN: All right, so I'm not going to list all of them
either.
But I can list some of the categories,
like social, productivity, shopping, games, communication,
finance, food and drink, health and fitness, transportation,
and finally common.
Common just means generic built-in intents
like, for example, if you want to get
a summary of your account or a check-in order
that you just recently placed, or just actually opening up
a feature in your app.
Now, with this in mind, a developer
might look at the list and say, I actually
don't think any of these built-in intents fit me.
What can they do in that case?
JESSICA EARLEY-CHA: Every Android app is unique.
And that's the best thing about the ecosystem.
And so what we have is custom intents.
And those are similar to built-in intents
where you could connect your Assistant to your app.
So what you first need to do with custom intents
is you need query patterns.
And these are the examples that are
different ways a user could say something
for that particular functionality.
So let's say I have a cake-making action.
And I want to give my users a way to say,
hey, Google, open baking fun and start making cupcakes.
I would need to provide several different ways for a user
to say something like that.
So someone could say something like, craft cupcakes
or produce cupcakes.
You can also pass in parameters.
So if a user says something like,
start making 18 cupcakes, that number 18 is passed
as a parameter, allowing you to provide
your users with instructions on making 18
cupcakes versus the usual 12.
And just note that parameters are
available in certain support types,
which include text, date, time, and number.
MARY CHEN: All right, I think go choose 18 cupcakes
over 12 any day.
JESSICA EARLEY-CHA: Right?
MARY CHEN: So let's talk about the technical details
around these new App Action functionalities.
Can you talk about the foreground app invocation?
JESSICA EARLEY-CHA: Yeah, of course.
So with forecast app invocation, this
allows built-in intents to be matched
without requiring the apps name to be mentioned
while a specific activity is in the device's foreground.
In other words, users can talk to the app
to quickly get things done.
And so if a user said something like,
order me a right Mountain View, the app
can use that input to set the destination
field to Mountain View.
You could also add this functionality
by updating your actions.xml file
and adding the required foreground activity attribute,
along with a new intent.
MARY CHEN: And a lot of app developers
actually have a strong web presence.
How can they make use of web inventory?
JESSICA EARLEY-CHA: That's a great question.
So let's start off with inventory in general.
So let's say you have about a boba restaurant app.
And a user can say, order a milk tea
from boba restaurant, which is great.
But for the Assistant to understand
that you have milk tea as a menu option,
you need to add this to your inventory.
You could use inline inventory where you list out
the different menu items as entities,
as well as their fulfillment URLs for each entity.
Now, for web inventory, you can use entity set reference tags
and URL filter attributes to generate the URLs
for fulfillment.
Web inventory is a great option for those with a strong web
presence.
And your in-app deep links are organized
around publicly available web content.
So now it's my turn.
So the last question before we move
onto the native developers.
While building new functionality,
being able to get support and to test functionality
during development is so critical.
Are there any tools to support Android developers?
MARY CHEN: Yeah, definitely.
So there's an Android Studio plug-in called the App Actions
Test Tool.
The Test Tool basically parses your actions.xml file
and then creates a preview of your app actions.
That way, you can test your app actions
without having to deploy the production version.
Also in Android Studio, you'll find
a lot of our documentation and support resources
if you need it as well.
All right.
So let's shift gears and talk about building
conversational actions for the Google Assistant.
Now, before we get into the questions,
Jessica is going to recap some of the major announcements
in this area.
JESSICA EARLEY-CHA: Yeah, so we announced
two new English voices for our developer community.
And this really takes advantage of the improved prosody models
and will sound even more natural.
We also announced recently in July
a new conversation tool called Actions Builder and actions
SDK.
These tools allow you to manage the conversational logic
in a state-based like modality.
This new tool has improved NLU and decreased latency,
as well as a new graphical representation
of the conversational paths.
Something else that was discussed today
is support for action links for Actions Builder.
So action links allow you to generate URLs
that you could use in marketing campaigns
that, when a user clicks on it, will
invoke your action on Google Assistant-enabled devices.
And something else we announced is
that we have two new beta programs to improve count
linking flows that will allow simple, streamlined,
authentication via apps.
The first one is linked with Google.
And the second one is App Flip.
And finally, if you have an action that's currently
being used and it uses Dialogflow,
if you want to use some of these new features
and convert over so you could use the newest tooling,
we've launched our migration tool
that will convert your Dialogflow
agent into an Actions Builder Project.
MARY CHEN: And I hope you saw it in the keynote today.
We also displayed the new learning hub
in stories lobby interfaces for Smart Slice.
If you missed it, these are cards
that display on your Smart Display
to help you explore and discover new storytelling and learning
actions.
Now, as a developer, I think my first question would be, how
can I get my content on there?
JESSICA EARLEY-CHA: Yeah, those were really cool, right, Mary?
Those hubs were amazing.
So a developer can actually get their content on there
by using implicit invocation for games, education,
and storytelling in their Canvas actions.
And so you can add this implicit invocation to your project
to the Actions Builder in Settings.
And just like your main invocation,
you have your implicit vocation transition
to the appropriate scene.
And you could also use Actions SDK to do this, too.
The great thing is if your action has more than one
of these, let's say your action is a game that is educational
where it tells you stories, you can use all three of them
if you like.
So you get multiple implicit invocations working for you.
So I have a question, Mary.
And I know that somebody had a lot of involvement with games.
So what type of games work best for Smart Displays?
And where could I learn more?
MARY CHEN: Yeah, we have to admit,
we probably planted this question
because we wanted to talk about it.
Over the last few months, we actually
took on the role of being game developers ourselves
just so we could use our tools to build quality
games for Smart Displays.
That came with a lot of learning from that development journey.
So we wanted to publish those as best practices,
lessons learned, and what not to do.
So as you may have heard, there's
a new assistant games portal.
This is our new website for game developers.
And on there you'll find these new sources,
for example, a new game design guidelines and case studies
for how we built these games.
And the main comment I want to make
is when you're building games for Smart Displays,
it's a really new and exciting space.
There is new interaction models where
you're using your voice as a primary mode of gameplay
and having these really rich graphics sit
in your players' homes.
But another way of thinking about it
is, you're just actually elevating your existing game
design expertise with unique capabilities from the Google
Assistant.
So there's a lot of familiarity with that as well.
And when you're building these games,
we used a couple of core Assistant technologies,
such as Interactive Canvas and the new Actions SDK.
Jessica, can you talk a little about how
these products work together?
JESSICA EARLEY-CHA: Oh, yeah, totally.
So Interactive Canvas connects the visual and the interactive
app with an action.
So you could think of your action
as the conversational interface that processes the user's input
and handles the conversation logic
and is built using Actions Builder or Actions SDK.
Now, your web app contains the visuals, the animation logic.
And that's built using HTML, CSS, and JavaScript.
And by using Interactive Canvas API,
you are connecting these two.
So for example, when a user says something like, turn the screen
blue, the action handles the user's input,
uses scenes to manage the conversation logic,
processes that, and generates a Canvas response,
along with metadata for the device.
And this metadata most likely will contain the color blue.
And in the web apps callback, when
it receives that metadata of changing the screen blue,
it actually will change the screen blue.
MARY CHEN: Now, one cool benefit of Smart Displays
is being able to combine visuals and audio really nicely.
How do you sync visual and audio elements
when you're developing for Smart Displays?
JESSICA EARLEY-CHA: Oh my, yes.
The worst thing is let's say you have an animation happening
and the noise happens later.
MARY CHEN: Oh, no, that's terrible.
JESSICA EARLEY-CHA: So what you could
do to make sure that these two things are in sync
is using ON TTS mark.
And let's say you have a storytelling action.
Every time there's a puppy character mentioned,
you want the image of the dog to do some animation and to bark.
And so you could have this functionality
by adding a unique mark name into your SSML,
like the word puppy dance.
Then that gets registered on the TTS method of the callback
object in your web app.
So every time your action receives
that mark name of puppy dance in your SSML, the animation
and the sound will happen at the moment of that TTS.
MARY CHEN: And finally, I hope this
is exciting for our people who are new to Interactive Canvas.
How can developers get started if they actually don't
have a Smart Display currently?
JESSICA EARLEY-CHA: Oh my gosh, this
is one of my favorite questions, because I'm always
amazed by how developers are so eager to build using the latest
tools, even if they might not have the devices.
And so you could use action simulator.
And the actual simulator simulates the interactivity
of your action.
And we provide different device types.
So if you personally don't have a speaker-only device
or a Smart Display or an Android phone,
you could still build for those users
and be able to test out and see and hear how your users will
be experiencing your action.
MARY CHEN: Yeah, and we personally used it all the time
when we're building out these games
to test on multiple devices.
JESSICA EARLEY-CHA: Right.
MARY CHEN: So check it out.
All right, so we covered a lot of questions here.
And I also finished my tea.
So I think we should call it a wrap.
Thank you so much for all the questions
from before and for the ones that you asked on the chat.
JESSICA EARLEY-CHA: Yeah, and if you
want to get connected to our community,
check out our Twitter account, Actions on Google.
That's where we post up all our announcements.
And join our Reddit community.
That way you can ask questions and hear
from other Action developers.
And if you want to start building and learning
how to build, check out our docs, our GitHub samples,
as well as our codelab so you can
get step-by-step guides on how to build for Google Assistant.
Thank you so much.
Really appreciate it.
MARY CHEN: Thank you.
[MUSIC PLAYING]