[MUSIC PLAYING]
XIN WEI: Hey, everyone.
My name is Xin.
I'm a software engineer working on Firebase Remote Config.
To start off, who here likes coffee?
Awesome.
Did anyone check out that fancy barista app out there
by the coffee bar?
Yeah?
Now, did you all know what you wanted
to order before you got there?
Well, I didn't.
I love coffee.
But it was my first time using the app.
And I was quite pleasantly surprised and a little blown
away by the sheer number of different drinks
and customizations it has.
I was staring and scrolling through the screen
for the longest time, trying to figure out what I wanted,
and how I wanted it.
I'm pretty sure there were people behind me in line that
were thinking to themselves, can he hurry up
and stop holding the line?
I kept wishing that a virtual barista of some sort
would magically just pop up and automatically
recommend something to me.
And wouldn't it be great if, as I order more and more drinks
in the future, the recommendations could just
improve, giving me a better experience each time?
Now, imagine you're developing an app
to provide that sort of experience.
Just like how everyone has different tastes when
it comes to their coffee, your users
may also have different preferences and expectations
when using our app.
They want their app experience to be
better than the last time they were on it.
And there are so many things that could reduce
the quality of that experience.
A misplaced ad, for example, it could annoy a user
into just closing your app in the middle of a big order.
And they may not come back.
And that's not so great.
But unlike a real-life barista, your app
can't simply just ask your users what kind of experience
they will like.
A barista may recommend something
based on subtle hints like customers' mood, their order
history.
And that often means using these complex psychological
inferences that our brain is capable of generating.
But how do you perform these kind
of complex, multifaceted analysis in an app though?
Well, with machine learning, of course.
Using the power of machine learning,
Remote Config Personalization gives you
the ability to create several user experience alternatives
and automatically provide the best one to each of your user
based on a metric code that you care about, like maximizing
user engagement or revenue.
And best of all, you don't have to be a machine learning expert
to do this.
You don't have to worry about setting up complicated
pipelines or models because Remote Config Personalization
does it all for you.
Now, before diving into personalization,
let me quickly tell you a bit about what Remote Config is.
You can think of Remote Config as a key value
store with powerful conditions to create targeted user
experiences.
So for example, a key can have several values
based on a language, a country, an analytics audience, or more.
Now, one of these special values can also
be a machine learning-driven personalization.
As you use Remote Config in your app,
the personalization algorithm will automatically
select the best value for each of your user
to maximize your objective.
So you can think of this as performing
an automated individualized continuously improving
A/B test.
And B, as time goes by, user behaviors might change.
And that's OK.
Because the personalization algorithm
will continue to learn, adapt, and adjust,
always trying to optimize for your objective
at any given time for a particular user.
And because we take care of everything for you,
all you have to focus on is just building the features
your users love.
Remote Config Personalization is built on Google Analytics.
And for those of you who haven't used Analytics before,
the two main features of Analytics
are the ability to measure predefined and custom
events in your app and report specific event parameter
values for important metrics surrounding those events
as they occur, so for example, an e-commerce purchase event
or e-commerce purchase amounts.
Remote Config Personalization gives you
the ability to use any Analytics event as an optimization
objective.
So let's say, for example, you have an event relating to,
I don't know, ad revenue.
And that it could have a numerical value attached to it,
let's say, the amount you earned for a click or impression.
Not only can you optimize for how often those ads were
clicked or shown, but also how much total revenue
they brought in.
This way, you can use personalization
to maximize the user lifetime value that comes from that.
And this is pretty cool.
Or at least, I think it's pretty cool.
So let's walk through an example on how to set that up.
So here, I have a mock of my amazing test app, the New Grind
Coffee.
And as you can see, I currently have
a banner app that's displaying near the top of my screen.
This is currently set by the fall value and my Remote Config
parameter.
So let's use personalization to, well, personalize
that banner app allocation for each of my user
to maximize my objective.
I think, for our little experiment here,
the four alternative options I want to explore
is top, bottom, inline, and bottom fixed.
So let's jump into my Firebase dashboard to set this up.
Great.
So I start in my Firebase dashboard.
And I can see the various Remote Config
parameters I have controlling different app
experiences in my app.
Let's click and zoom into one of them--
ad placement, which you probably have guessed,
it controls the placement location
of that banner app in my app.
As you can see, it's currently set to a default
value of top for all my users.
I think each user responds to ads differently.
That's what makes ad revenue so hard to optimize for.
I want to put my ad at the right location for each of my users
to maximize my objective.
But I don't really know where that location is.
I don't want it to be too disruptive for the user
that they'll get annoyed and just leave my app before I even
place an order.
But at the same time, not so subtle that they'll completely
ignore and miss the ad, which is pretty common for banner ads.
So let's use personalization in Remote Config
to figure out the best ad placement
location for each user.
So to personalize a parameter, I click on Add New and click
on personalization.
This will bring up the personalization flow.
The top section that you see here
is where you can define several alternative values
that your users can have.
So in our case here, that would be the different placement
locations for my ad that I can have my users experience.
So as mentioned before, let's explore the four options
here, which is top, bottom, inline, and bottom fixed.
Cool.
Next up, I need to select my objective.
Since I want to maximize for my total ad revenue,
I'm going to use personalization to maximize the total parameter
value of a custom Analytics event I have set up.
So I have a custom event set up called ad revenue.
I know, it's not very original.
But attached to that is a flow type parameter called value.
Now, in my app, I'm logging this event.
I'm forwarding the impression level AdMob revenue
that I'm getting from AdMob into this parameter.
And I can verify that everything is working
as expected by heading into my Analytics event dashboard,
and select my event, and verify that my event
is showing up here with the right parameter
and values populated.
Everything looks pretty good.
So let's go back to my setup.
Great.
I can see that my custom ad revenue UN is showing up here.
So let's select that.
Next, since I want to optimize for the sum of my total paid
impression, I need toggle some there.
And then, I enter the name of the event parameter
that I'm forwarding my AdMob data to.
So that will be a value.
I'm also interested to see what happens with user engagement.
But I don't want it to be driving the optimization
algorithm.
So let's select that as an additional metric.
I'm almost done.
Next, I need to select a target segment.
Let's target all my USA users for now
and see how that works before potentially expanding it
to everyone.
Give it a name and a description,
and pretty much done.
Click Save and Publish in order to get
this personalization running.
Great.
As you can see, I'm now waiting for personalization
to work its magic.
This can take some time.
So let's jump into some tips and tricks on how we can set up
an effective personalization.
As magical as machine learning is,
it doesn't happen overnight.
And it takes time to build and improve the models.
While you should be able to see some results in a day or two,
we recommend just leaving it run indefinitely.
That way, personalization can continue
to learn, ensuring that your app stays in sync with your users
should their preferences change over time.
Some of the most effective personalizations
that we have seen are the ones that
have been running for at least a week, usually much longer.
When picking a targeting condition,
choose a large enough target segment
that will generate plenty of result data,
ideally with as many users as possible.
When picking alternative values, pick
ones that produce a large difference in outcome.
The system will learn faster and create more value
when the alternatives are less similar to each other.
And finally, good machine learning models
thrive on lots of data.
So when picking a optimization objective,
choose an event that fires pretty frequently.
Cool.
And as shown before, you can use the Analytics event dashboard
to verify that the event you're trying to optimize for
is being fired.
If you're not getting any data in your personalization
after a while, this is most likely the issue.
If you're using custom parameter value maximization,
make sure that the event and parameter names match exactly
with what you see on your events dashboard
and that it has a numerical value attached to it.
You can also search for personalization assignment
events.
These events are on active fire whenever a user gets a value.
It notes the personalization ID, the group that the user is in,
whether it's baseline or personalized,
the alternative index and value that the user got,
and the Remote Config parameter name that's associated
with this personalization.
So what can you do with all this data?
Well, you can export it to BigQuery
and use this event to identify users
in each group of interest and slide share Analytics data
accordingly.
Cool.
Now, it's been a while.
So let's go back to our personalization
to see how it's doing.
After about a week or two, the personalization
should be working pretty effectively.
So to see the results, I just need
to click on the personalization icon, which
is the sparkly person icon and my Remote Config parameter,
and wow.
I am pretty impressed that my personalization
is outperforming my baseline by 8%.
This is pretty magical.
So let's dive into the results a little more.
So the top section that you see here
shows the high level information about my personalization,
including when the personalization was started,
the objective I'm trying to achieve,
my targeting condition, and the various alternative
values that my users can have.
You can also see how many users are
exposed to my personalization on the top left.
The bottom section shows my performance.
That baseline group, which is the gray bar chart here,
is a group of random users that will
receive a persistent non-personalized value.
It gives me a way to understand and compare
what I'm getting from using personalization.
Now, that baseline is only 20% of my total users.
So it's way smarter than a personalized group.
So it's scaled up here to be comparable.
Now, to see my performance over time,
I just need to head to my top right
and click on the daily toggle.
And as you can see, my performance
started out as not super great for the first few days.
But it got better over time.
And this makes sense.
Because the personalization algorithm
will continue to learn, making smarter
and better decisions the longer it runs.
Now, to toggle between my overall objective
and the secondary metrics I'm interested in,
I just need to click on the top.
And here, I can see that my user engagement time is not too
impacted by my ad placement.
And that's exactly what I want to hear.
Personalization is exceptionally powerful
when different users benefit from different experiences.
For example, setting a dynamic gain difficulty level
or figuring out the right discount
percentages for the promotions that you are running.
So go ahead and give personalization
a try to give the best version of your app
to each of your user.
If you're already using Remote Config today,
you can start creating personalizations
in your dashboard.
If not, here are some videos and documentations
to find out more about what Remote
Config and personalization can offer,
including how it's different from our A/B testing products.
Cool.
That's all for now.
And if you have any questions, you
can find me in the Ask Firebase lounge area after this,
probably with a coffee in my hand.
Thank you.
[APPLAUSE]
[MUSIC PLAYING]
[FIRE CRACKLING]