[MUSIC PLAYING]
MELISSA LOPEZ: Hi, I'm Melissa, a software engineer
on Firebase.
MAPLE KUO: And I'm Maple, a UX designer also on Firebase.
MELISSA LOPEZ: We're here to talk to you about monitoring
your latest release.
Building and releasing new features
is critical to app development.
But launching a new release can be scary,
especially if it's for a major feature in your app
or a totally new version of your app altogether.
Leading up to a launch, your team
can have a variety of priorities.
For a brand new feature, maybe you
care most about ensuring that the user experience is
intuitive and that users aren't crashing at critical points.
For an improvement to an existing feature,
maybe your team cares most about technical performance,
like the response time of a new API endpoint,
because of the impact performance metrics
have on app revenue, retention, and churn.
And while releases are an exciting time
to engage your users by bringing them more value,
it comes with a real risk of introducing clashes or slowing
your app down.
We know you have a lot of things to worry about already.
That's why we've been hard at work building features
to help you focus on fixing problems rather than finding
them.
Let's imagine that you're working on a shopping app.
And your team is releasing a new checkout experience.
As part of this new flow, you've adopted a third party checkout
solution.
And the engineering team has five top priorities, ensuring
that users aren't crashing, that alerts set up
to notify you if customers are crashing,
checking if the new feature has introduced any new crashes,
that app start time is still fast,
and because your checkouts are being handled by a new API
endpoint, you want to make sure that response time remains
low for the core check out API call.
This can seem like a tall order.
But don't worry, we've got you.
We'll walk through each of these in detail
and guide you in monitoring your new release
to ensure a successful launch with Firebase Crashlytics
and Performance Monitoring.
MAPLE KUO: Let's start with checking
how many users are crashing in your latest release.
You want to make sure that it's pretty stable.
Let me pull up the Crashlytics dashboard so we
can check on crash free users.
So by default, the Crashlytics dashboard
shows you information for all versions.
But since you're focused on the new release,
let's go ahead and filter the dashboard by that version.
All you have to do is click the Filter button at the top,
select the latest version, then hit Apply.
You can see here that crash free users
are above our target goal.
And if you want to check how many users have adopted
your new release, you can do that
by clicking here and going to the latest release page.
OK, so we just finished checking on your app stability.
And things are going pretty well.
But what if things aren't going well?
What if there's actually a big issue that's
affecting a lot of customers?
It's important to have alerts setup so you get
notified as soon as possible.
One of the most important alerts in Crashlytics
is velocity alerts.
Velocity alerts are triggered when 1% of user sessions
are experiencing the same crash within an hour.
AKA, whenever there is a high impact
issue happening in a short amount of time,
you'll get a notification.
You don't even have to set anything up.
To save you time, we've automatically
enabled velocity alerts in Crashlytics,
meaning you get them sent straight to your inbox
without any setup.
And if you want to get alerted even sooner
than 1% of sessions, you can always customize the threshold.
Since the checkout flow is expected
to drive a lot of traffic to your app, let's
lower the threshold.
Let me show you how to do this.
In Crashlytics, click on the overflow menu
on the issue table, then select Velocity Alert Settings.
You can see that the threshold is currently set to 1%.
That's the Crashlytics default. To change the threshold,
just drag the slider.
Since we want to get notified about high impact issues
right away, I'm going to lower the threshold to 0.1%.
And then I'll click Update Thresholds to save my changes.
This means that you'll get a velocity alert notification
as soon as a crash is affecting 0.1% of sessions
within an hour.
Now you might be thinking, well, I'm not inbox zero person.
That alert is probably going to get lost.
It'd be nicer if I get this in Slack.
Well, we've definitely heard that before.
And that's why we made a Slack integration for Crashlytics.
We also have integrations with Dura and PagerDuty
so you can get notified in the tools
that your team is already using.
Now with the Slack integration, you
can set it up to send a notification to any channel
when an alert is triggered.
For example, you can set up a velocity alert trigger
to post to a Slack channel called latest release.
Here's what the notification looks like.
To learn more about the crash, just click on the issue name.
That will take you to Crashlytics.
You can also set up notifications for other alerts,
like regressed issues.
For more info on setting up alerts and integrations
in Crashlytics, check out the link in the description below.
Now that I've covered some alerts,
we can go ahead and mark this one as done.
The next thing we'll be checking is
how to find crashes in the checkout flow.
With any new release, your app might
be crashing in a few different places,
this might be the checkout flow or it might
be the onboarding experience.
Let's say that you're really interested in knowing
if your app is crashing for any customers who have bought
something, since a big reason why
you shipped the new checkout flow is to increase revenue.
In Crashlytics, you can see all the crashes
that are happening in your app.
But it can be hard to see which crashes
happen to just customers that bought something.
Luckily, there's a way to do this
with custom keys and BigQuery.
Let's start with custom keys.
Now you might be wondering, what is a custom key?
Well, a custom key helps you track
the state of your app leading up to a crash.
Another way to explain this is that a custom key
provides a snapshot of information
and records the last known value.
Here's an example.
If you're a game developer, you can use a custom key
to track which level a user is on before they crashed.
And if you're developing a shopping app,
you can use a custom key to track
if a user has bought something or how many items
were in their cart before they crashed.
We want to see crashes for customers
that bought something.
So let's go ahead and instrument a custom key for that.
To set up custom keys for iOS, use the Set Custom Value
instance method to set key value pairs.
For Android, use Set Custom Key.
Here's an example of how you might set up your key value
pairs.
After you've set up custom keys, they
will start showing up for crashes in the Crashlytics
issue detail page under the keys tab.
This is great if you've already identified an issue
and need more info on how to fix it.
I did mention earlier that you can
get some kind of dashboard that lets you see crashes
by specific parts of your app.
We just did the first step where I
walk through how to use custom keys to tag
parts of the app you're interested in.
The second step is to set a BigQuery.
BigQuery gives you access to your raw Crashlytics data
through an integration.
So you can get more in-depth crash reporting
and run custom queries.
You can even set up ways to automate your release process.
For example, you can run queries to see
if any crashes in your beta and production releases
cross a certain threshold.
You can also run a query to see crashes by custom keys,
like the one we just added.
So how do we do this?
To start, let's enable the BigQuery integration
in Firebase.
All you have to do is go to the integrations tab in settings,
select BigQuery, then follow the steps to add the integration
for Crashlytics.
And even if you already use BigQuery with Crashlytics,
you'll be excited to know that we recently
launched the ability to export data in real time.
We know a lot of you have been asking for this functionality
and we're really excited to see how real time will power custom
workflows and alerts for you.
To enable real time, just check off
include streaming in your BigQuery settings
for Crashlytics.
After you've set up the BigQuery integration,
you can run queries on your raw crash
data, which means you can find exactly
which crashes are happening for custom keys that you've added.
Now this can sound a little intimidating.
And you probably want to get back to building your app.
So to save you some time, we've made a Crashlytics data studio
template that does the heavy lifting for you.
That way you can focus on fixing the crashes
instead of finding them.
The template is powered by your exported crash data
from BigQuery.
There are four different pages and each one
focuses on a different use case.
Your crash data can be sliced a little differently
from the main Crashlytics dashboard, which is useful
if you want to see things like crashes by device,
crashes by keys, crashes by file, or real time trends.
The template provides a solid starting point for your team.
And you can always customize it if you need to.
To use the template, click the Use Template
button on the top right to link your BigQuery export data.
I've added the template and some useful links in the description
below.
Now let's go back to our earlier goal
of seeing crashes for customers that have bought something.
We'll do this with the crashes by keys page.
This page lets you see all the crashes happening
for any custom keys you've added.
Just use the filters at the top to select your custom key.
You can also select a specific version
or adjust the date range as needed.
Once the dashboard is done loading,
you can see all the crashes for customers
that have made a purchase.
This it's great to help you prioritize what's
most important to fix first.
And if you want to learn more about the issue,
clicking on the name takes you straight
to that issue in Crashlytics.
And if you're using BigQuery streaming,
new crashes will also show up here in real time.
All you have to do is refresh the page.
To recap, you can check on specific crashes
in the checkout flow or any feature you've
added by instrumenting custom keys in Crashlytics
and leveraging BigQuery in our Data Studio template.
Use these tools to monitor what's most important for you
and catch crashes earlier so that you can
start working on fixes sooner.
MELISSA LOPEZ: Now let's dig into app performance,
starting with our next key metric for this release, app
start time.
App start time is a measure of time
between when the user opens the app
and when the app is responsive.
When you integrate the performance monitoring SDK
into your app, you don't need to write any code
before your app begins to monitor app start time
and other performance metrics automatically.
Better yet, the newly revamped performance
monitoring dashboard makes keeping track of app start time
easier than ever.
Let's take a look at this in action.
The performance monitoring dashboard
has been freshly upgraded, providing
a glanceable and customizable overview of how
your key metrics are trending.
For mobile apps, app start time is
included on the dashboard right out of the box.
On the new dashboard, we can see how
the latest version is performing compared
to your app's baseline.
We can immediately see that app start time is not
where we want it to be and can start working on improvements
right away.
In the meantime, if your app also uses Firebase Remote
Config, you could turn the new feature on or off
without having to redeploy the app.
This would allow your app development team
to work on fixes without some of the added
pressure of knowing that users are having
a bad experience at app start.
If you want to learn more about rolling out
new features with Remote Config, feel
free to check out the link below.
When the team's done, you can come back
to the dashboard to validate if the fix is
working for real users.
And if the dashboard starts trending green,
the fix is working.
Now that this metric is on the performance monitoring
dashboard, a quick visit to the dashboard
is all you'll need to be able to tell
if app start time is or isn't performing as it should.
Having app start time out of the box is great,
but what if we want to measure something
more specific to our app?
For example, what if we want to gauge how easy
the checkout flow is to navigate,
such as by measuring how long the checkout process takes.
To do this, we can leverage custom traces.
Custom traces are a bit like timers.
You can start and stop a trace at any two points in your code
to measure things like the duration of a task,
such as how long it takes a customer
to complete their checkout, or to count things
like cash hits or retries.
To track the duration of the checkout flow,
it's as simple as including two lines of code.
The first line of code is to start
the trace, which we'll put at the very beginning
of the checkout flow.
And the second line is to stop the trace.
We'll stop the trace when the user hits the button
to place their order.
And that's all the setup it takes
to get this metric reported in the Firebase console.
Next, you'll recall that the new checkout flow is issuing
requests to a new API endpoint.
For these new requests, our team really
wants to be sure that network response time stays low.
Performance monitoring reports metrics
for any endpoint to which your app makes a request.
These reports are called network request traces
and they capture the time between when your app issues
a request to when the response from that endpoint is complete.
Some of the captured metrics include response time,
payload size, and success rate.
Performance metrics for similar network requests
will get aggregated into what we call automatic URL patterns.
These are helpful for being able to understand trends
in your network requests performance.
But in our case, we have an exact high priority request
to monitor.
So we want to make sure that insights won't get buried
in an automatic URL pattern.
To do this, we can create a custom URL pattern
to capture the specific endpoint.
With custom URL patterns, performance monitoring
will try to match network requests URLs
to any configured custom URL patterns
before falling back to automatic URL pattern matching.
So by creating a custom pattern that
matches requests made to our new endpoint,
Firebase will aggregate the request data
under the new custom URL pattern.
To learn more details about URL pattern aggregation,
check out the docs which are linked in the description.
Custom patterns can be created from the Network tab
in the performance monitoring console.
Here you'll find a big blue Create Custom Pattern
button, which opens a dialogue for you to input your pattern.
You'll notice that the dialog provides
some examples to teach you about the syntax
rules for custom patterns.
These options are helpful depending
on whether you want a custom pattern to exactly match
a URL for a particular call or if you
want to aggregate metric data for a bunch of network calls
under a single pattern.
In our case, we want our pattern to capture one exact URL.
So that's what I'll type into this text field.
Once this pattern is created, it'll
appear in the Custom tab of this table
and will look like this until it starts getting data.
Once there's data, we'll be able to see if response time is
where we want it to be.
And it looks like currently it is.
But say you want to go the extra mile
and keep a close eye on this over the coming days.
This metric can be added to the dashboard,
just like other metrics.
I'll go ahead and add this metric.
Now it only takes a quick glance to keep tabs
on how response time for this call is trending.
With that, we mark another metric green
on our launch checklist.
Now our dashboard has exactly the key performance metrics
that we care about for this launch.
And if we ever want to change the dashboard,
we can always add or replace any metrics later down the road.
And because it's so important to spot issues and validate
improvements quickly, we're delighted to share
that we'll start reporting performance metrics
in real time coming soon.
And with that, we finish walking through our top tips
on monitoring your latest release.
To recap what we covered, we started
by checking on the stability of the latest release
with crash-free users in Crashlytics.
We talked about how to leverage important Crashlytics
alerts, like velocity alerts, to get notified of issues sooner.
And we were able to identify new crashes in the checkout flow
with custom keys, BigQuery, and Data Studio.
With performance monitoring, we customized the dashboard
with key metrics to monitor app start time
and to validate improvements made by the team.
And we tracked the latency of a new API call
in the checkout flow.
We covered a lot of ways to monitor your latest release,
but even just a couple of these techniques
can go a long way towards making your app more stable.
Because after all, a successful app is a stable one.
While you're working hard on your app,
Firebase will be here to support you.
Best of luck with your new releases.
[MUSIC PLAYING]