POLONG LIN: In this video, we'll show you
how you can predict user churn for your gaming apps
using Google Analytics 4 and BigQuery ML.
My name is Polong Lin, and I'm a developer advocate
for Google Cloud.
MINHAZ KAZI: I'm Minhaz Kazi, and I'm a developer advocate
for Google Analytics.
If you're a mobile game developer,
a potential challenge for you could be user retention.
With so many games and apps to choose from,
keeping users interested in your product
and enabling them to come back can be really difficult.
According to a study in 2019, most mobile gamers
see only 25% retention for users after 24 hours.
That means on average, three out of four users will churn.
They'll first use their app, and then after a day,
they'll stop playing.
So the question is, if I'm mobile game developer,
how do I reduce churn?
Polong, you have experience with machine learning.
Can that help here?
POLONG LIN: Absolutely.
At a high level, you can use machine learning
to predict which of your users are going
to churn based on how they're behaving
in the app and their demographic information.
Once you have the predictions of who's likely to turn,
you can then target those users who are perhaps on the fence
to incentivize them to come back to the app or game,
or make the game just a bit more rewarding for them to play.
Predicting each user's propensity to churn,
that's where machine learning can help us.
MINHAZ KAZI: User's propensity to churn.
That sounds interesting.
Maybe we can apply this to a real life example.
We have a small little game called Flood-It!
that's available both on the App Store and Google Play Store.
Can we identify users' propensity
to churn for that app?
POLONG LIN: Certainly.
As long as you have the behavioral data,
you can build machine learning models
to predict user propensity to perform any specific behavior
like churn, or making in-app purchases,
or sharing the game with a friend, just as examples.
MINHAZ KAZI: So here's a scenario.
We have this Flood-It! app and we
can use usage data from there.
My goal is to identify specific users who
are likely to churn and then try to retain them.
I can, for example, through Firebase,
send notifications to these users,
or customize the game to be extra rewarding to them.
Through Google Analytics, I can segment the users
and analyze different likelihoods of churn
and do marketing campaigns, or through other marketing tools
tie together with email or other marketing campaigns.
So Polong, how am I going to use machine learning to do this?
POLONG LIN: Hold on.
There are a couple of steps in order
to get to these activation scenarios.
Let's walk through all the required
steps in reverse order.
So you had just talked about the last step, which is using
the predictions for activation.
In order for that to happen, you'll
need a churn prediction model to give you
these predictions on each user's likelihood,
or probability to churn.
And you can build this machine learning model
using BigQuery ML.
But to create this model, you'll need
to first have prepared the data in the right format
for the machine learning model.
So we call this step data preprocessing,
which is done in BigQuery.
So that brings us to the first step,
which is making sure you have your data ready in BigQuery.
MINHAZ KAZI: Great.
That fits in nicely since we can use Google Analytics
4 to measure usage data for the Flood-It! app.
We can just export that usage data into BigQuery.
POLONG LIN: Minhaz, could you tell us a little bit
about Google Analytics 4?
MINHAZ KAZI: If you're not already familiar,
Google Analytics 4, or GA4, is the newest
edition of Google Analytics.
It allows you to measure user engagement,
not just for websites, but also for apps.
GA4 uses an event-based model.
Usually, when the user or the app does something,
that's an event.
If you're an app developer who's already
been using Google Analytics for Firebase,
you can be rest assured that it's
the same data model as GA4.
What's new with Google Analytics 4
is that any user, free or paid, can now export the GA4
data into BigQuery.
POLONG LIN: And exporting data to BigQuery
is really interesting, because with BigQuery,
which is the data warehouse on Google Cloud,
you can analyze terabytes of data in seconds.
And you can also train and deploy machine
learning models, as you'll learn in this video.
So if you're a marketing analyst,
you can train and deploy machine learning models
just by writing SQL without any additional programming
languages.
So Minhaz, how would a Google Analytics user
export their data to BigQuery?
MINHAZ KAZI: I can log onto the Google Analytics 4 property
and go to the Admin section.
Then click on BigQuery Linking.
On the right hand side, we will see that, for this property,
no link has been created to BigQuery.
So we can click on link.
Then it'll ask for which BigQuery or Google Cloud
Platform Project to link it to.
If you haven't used BigQuery before, at this point,
you'll just need to set up an account on Google Cloud
platform and select the appropriate project.
We'll select our project.
And then click next.
For our analysis, we'll use the daily export.
After you've reviewed and submitted your expert options,
GA4 will start exporting the event data for this property
to the selected BigQuery project.
After a day or two, you should start seeing the daily event
data in the events table.
In the BigQuery UI, we can click on the Preview tab,
and it'll show us what the event export data looks like.
It shows you the unique event that occurred
at a specific time stamp.
You also get the event name and additional information
about the event and event parameters.
For example, in the first row, it
looks like the event was a level complete quickplay by the user.
So now that we have our GA data in BigQuery, Polong,
what's our next step?
POLONG LIN: Well, in order for the machine
learning algorithm to train the model properly,
we need to pre-process the raw GA4
data to be in the right format.
This is an example of what an ideal format might look
like for the training data.
Specifically, every row need to be
represented by a single user along with a label of churned 1
or 0.
We also want to include features that can help the model learn,
specifically the user's demographic information,
and any sort of behavioral data such as the number of times
they open the app within a certain time period.
So the way this works in this data preprocessing stage
is, first, we'll need to identify users
and the churned label, process demographic features,
and process behavioral features.
Then, we'll combine them together as the training data.
With the training data ready, we can then
move on to building the churn prediction model.
So the first step is making sure we have
a solid definition of churn.
Now, your definition of churn may vary here.
You may want to predict churn after one day, one week,
or simply whether or not they will churn up to the first 30
minutes of installing the app.
And that definition is entirely up to you
to decide what makes sense for your app.
For us, we chose to define churn as the following.
After 24 hours of a user's first engagement with the app,
the users who never came back we called churned.
Those who come back after 24 hours, we can say they
returned.
That means, given all the user data
that we have, we want to say, based on the user information
within the first 24 hours of them using the app,
can we predict if they will churn or return?
MINHAZ KAZI: But hold on, what about those users who
use the app and then within just a few minutes
they decide that it's not the app for them at all?
Perhaps those users aren't worth targeting
with any sort of incentives, because this wasn't something
they were looking for.
I would prefer to exclude them.
POLONG LIN: That's a good point.
Let's ignore users who left the app within the first 10
minutes.
We'll call these balanced users.
So when we build the machine learning model,
we'll only look at users who didn't bounce.
MINHAZ KAZI: So the prediction model
will be based only on the users who spend
at least 10 minutes on the app.
To prepare the user data in BigQuery,
first we'll create a bounced label column.
Engagement is an event when the user interacts with the app.
If the user's last engagement was
within 10 minutes of their first engagement,
we'll consider that the users have bounced.
Bounced users will get labeled with the 1
in the bounced column.
Otherwise, the value is 0 for all and most users.
Next, we'll create a label column for churned.
So if the user's last engagement was
within 24 hours of their first engagement,
we will consider the user to have churned.
Otherwise, we know that they did something with the app
after the first 24 hours, meaning
that they did not churn.
So we give those users a value of churn equals 0.
So this is what it looks like now.
One label for churn and one label for bounced.
POLONG LIN: Perfect.
So in the next section, we want to process
the demographic information available for each user.
Minhaz, can you explain what demographic data
is available in the GA4 data?
MINHAZ KAZI: In GA events export,
some demographic data about the user
is recorded automatically, like the device
language and operating system, user's browser,
and geographic information.
You can also use user parameters in GA4 to capture
additional user demographic data.
For simplicity, we only chose country, operating system,
and the user's device language.
POLONG LIN: And here, if there are multiple rows per user,
like if they were in different countries,
then you can just use perhaps the most recent
demographic information to have only one row per user.
So the next step is preparing the behavioral data.
And this step is really important
because you want to help the machine learning model learn
how the user's behavior in those first 24 hours
predict whether or not they churn.
Minhaz, what sort of behavioral data does GA4 capture,
and what fields will you consider using?
MINHAZ KAZI: As we mentioned before, each row in GA4 data
is an event.
Some events are automatically captured by GA4.
There are recorded events for specific use cases like gaming.
You can also create custom events, depending on your app.
Let's see what type of events we have in the flooded data set.
We'll query all the event names and sort by the account
across the data set.
As you can see, there are automatically
collected events like screen_view
and user_engagement.
The other events are recorded events for gaming,
like level_start_quickplay and post_score.
From the list, we chose this subset of events
to be most important for our analysis.
But you can choose whichever events
you think might help the ML model determine who is
likely to churn for your app.
POLONG LIN: So we'd like this sort of result
where each row is a unique user ID, and for each user
we have some behavioral stats, like how many times they engage
with the app, how many times they
started a level with quickplay or ended a level with quickplay
and so on.
MINHAZ KAZI: We can achieve this by counting different events
in different columns for each user.
POLONG LIN: Now that we've prepared
our label, the demographic data, and behavioral data,
we can combine them into a single table, which
becomes our training data.
MINHAZ KAZI: A quick side note before moving on
to machine learning.
Let's assume that you have other demographic or behavioral data
from outside of GA that you want to use.
For example, the data could be in your CRM.
You can import that data into BigQuery
and then join them using the user ID,
and use that for training as well.
POLONG LIN: So with the training data ready, now
we can finally talk about building
the machine learning model to predict
churn using BigQuery ML.
In this section, we'll be using BigQuery ML
to train the churn prediction model, evaluate the model,
and make predictions on new users
to get their probability of churning.
If you're not familiar with what BigQuery ML is,
it's the ML, or machine learning functionality,
that's built into BigQuery by default.
So if your data is stored in BigQuery, without needing
to move it out, you can use BigQuery ML to train and deploy
machine learning models directly using SQL.
You don't need to know Python, R, or TensorFlow.
You also don't need to set up any extra infrastructure
because it uses BigQuery's auto-scaling infrastructure
to train and deploy your ML models.
So that means you get your data storage, data analytics,
and machine learning all built into BigQuery.
There are lots of different machine learning model types
available with BigQuery ML.
For propensity modeling, you'll want
to use one of these supported classification models.
To keep things simple, we'll stick
with logistic regression in this video
as it's typically the fastest to train.
You can always try to improve your models afterwards,
but this gets you to a working model quickly.
So to train a logistic regression model using SQL,
first, you'll need your training data.
Select star from the table you just put together.
Next, you'll want to create a replace model followed
by the project and data set and the name of the model, which
we've chosen here to be churn_logreg.
Next, we'll need to specify that we
want to use logistic regression as well as setting
the churned column to be the target variable that we
want to predict.
And that's all you need.
And once the model is trained, you
can evaluate it in BigQuery ML.
And to do so, you can use the BigQuery UI,
which shows you the evaluation metrics
when you select the model you've trained.
You can see the metrics like log loss, ROC AUC,
precision, recall, accuracy, F1 score, and some visualizations.
You can also evaluate the model using select star
from ML.EVALUATE and then the model name.
If you create multiple models, you
can use something like ROC AUC or F1 score
to compare your models, with better models having scores
closer to 1.0.
And depending on your needs, this
is where you could be iterating between training the model
and evaluating the model to try to improve your model
performance.
There are various approaches to improving models.
You can try a different model type,
such as boosted tree classifier, deep neural networks,
or even auto ML tables, all which
are supported in BigQuery ML.
You can also try to select different features, or more
features, or engineer new features out of existing ones.
And, of course, you can also try tuning
some of the hyperparameters of your models.
To learn more, check out the documentation.
MINHAZ KAZI: Great.
So now with the model created, how
do we make predictions on who might actually churn or return?
POLONG LIN: You can use select star
from ML.PREDICT followed by the model and the data you
want to make predictions on.
Note that, for the test data, it should
have the same columns as you had in the training data.
We can see what the model predicted in the first column,
but this isn't actually what we are looking for.
For propensity modeling, we actually
want a probability of how likely someone is to churn,
some value between 0 and 1.
So for this, we'll need to extract out the probability
from the third column called predicted_churned_probs.prob
for when the label is equal to 1.
To do so, you can extract the probability of churn
for each user using this SQL query.
Now that we have the user's propensity to churn,
how do you use these predictions for activation?
Minhaz?
MINHAZ KAZI: To get the data into an usable state,
we can either-- we did directly from BigQuery,
or export it into Google Cloud Storage, or GCS, as CSV files.
This query will export the selected columns
into a CSV file in a GCS bucket.
We can use either the BigQuery data, or the exported CSV file,
in different ways depending on how the [INAUDIBLE] works.
For example, Firebase can directly import from BigQuery,
whereas external marketing tools might need to use CSV files.
In GA4, you can use the data import
feature to import the predicted values as a user property.
And then the user property can be
used to create segments and remarketing lists.
If you're using Firebase for your app,
you can also import the predictions back into Firebase
to tailor the user experience using Remote Config,
or send notifications to the user using Cloud
Messaging and In-App Messaging.
You might also use other external tools
to run marketing campaigns, like sending out emails
to motivate users to return.
Using the steps described in this video,
we should be able to do the activations
and finally increase the number of users retained
for the Flood-It! app and reduce the number of users churned.
POLONG LIN: Great.
So to recap, we showed you how to export your GA4 data
into BigQuery, how to pre-process
the churn label, demographic data, and behavioral data
to get the training data in a proper format in BigQuery.
Then, we talked about how to train, evaluate and make
predictions, using BigQuery ML.
And finally, we covered a few ways
to use the predictions for activations.
If you'd like to follow the steps,
you can check out the blog post, or the GitHub notebook,
by going to the links in the video description below.
Thanks for watching.