[MUSIC PLAYING]
DOUG STEVENSON: Hello, everyone.
Thank you for joining us here today.
I hope you're enjoying the event so far.
My name is Doug.
I am a developer advocate with the Firebase team.
IBRAHIM ULUKAYA: And I'm Ibrahim from the Firebase team as well.
DOUG STEVENSON: And we thought it
would be good to tell you about how you can use Firebase
products to test your app.
Now, the title of this talk is "How can Firebase
improve the user experience of your app?"
And I thought it might be nice to tell you about that.
But it's even better, I think, to show you.
And so that's what we'd like to do today.
We'd like to show you how we used Firebase in an app
that we built called FriendlyPix.
Now, FriendlyPix is an app that you
can use to share pictures with friends,
kind of like a social network.
It's open-source for iOS, Android, and the web.
We use it as a demo app to showcase Firebase APIs
and products.
Now, like all apps, there are problems during development.
We experienced this.
Ibrahim, remember that time when the cloud
storage for Firebase SDKs changed and we
had some problems with that?
IBRAHIM ULUKAYA: Yes, I remember that, Doug.
Actually, that was a strange time.
We were just ready to publish our first apps
And the creatives were successful,
and we just want to give it a try through Test Lab.
And Test Lab lets you test your app
in the wide variety of devices and Android and iOS.
They are hosted on the cloud.
Let's see on our app.
While we were testing, we realized
that we got an error on the test upload photo.
And this test was failing.
And it was pretty weird because we
didn't see this issue while we were testing ourselves.
So in this case, we can even see the logs and some video
here from the test.
As you can see here, the app is starting.
And you pick a photo, and the test is going through,
writing your caption.
And you've clicked upload picture,
and the upload is happening right now.
And it seems like it's going well but--
our app crashed.
So that was weird that our app crashed
but it wasn't crashing before.
And looking at this, we realized that it was in the upload site,
in the upload view controller.
So let's go back to our code.
And we realize that the download URL was a property
before but it was deprecated, and now it was in a sinkhole.
And as soon as we fixed the error,
our test was working as normal.
And since then, we are using the XCTest,
and we created some more UI tests as well.
So we go through our app, and we have a reliable [INAUDIBLE]
our app always works.
Let's see one of our tests, which was on an iPhone X,
I did, like, few days ago.
As you can see, it is accepting some permissions,
going through privacy, going through the authentication,
going through.
And it's taking a photo, uploading it.
Now, we're going to go to upload screen, writing your caption,
upload it.
And we will [INAUDIBLE] that.
And it is a comment, "saw test was successful."
This was a particularly close save,
and it was a really good save for us.
Because if we shipped our app and it
would crash in the very first few users,
that will be a terrible experience.
And it really saved us from some one-star reviews from that.
DOUG STEVENSON: Yeah, for sure, early one-star reviews
in your app would be bad.
It would certainly tank your ratings in any market.
So obviously, it's crucial to test
your app before you publish it.
Some of the members of the Test Lab team
say you should test in the Lab, not on your users.
Now, this is obvious advice.
So Ibrahim, I'm curious, how many iOS devices
were you able to test on?
IBRAHIM ULUKAYA: Oh, right now, actually, we had, like,
12 iOS devices on the Test Lab.
And we [INAUDIBLE] iOS versions as well.
DOUG STEVENSON: OK, yeah, and on--
Now, I'm an Android developer.
I've done a lot of Android development.
And I know all too well that it can be very challenging to test
on a wide array of devices.
You know, landscape, portrait, tablet, phone, Android TV--
all these different devices, it's challenging to do it well.
And it's good for consumers to choose.
They should choose the devices that
fit their budget and lifestyle.
I think, in Test Lab, there's something like 40 devices
to choose from.
And there's new ones being added all the time.
Now, Ibrahim showed you that you can write
an XCTest for your iOS app.
Now, it's worth noting that on Android, the similar framework
is called Espresso.
So you can write test to sort of remote control
your application.
But if you're not into writing code for your tests,
you can still use something called Robo Test.
So Test Lab has a fully automated, intelligent crawler
that will try to exercise the most of your app as it can.
And you can see right here how it's sort of automatically
clicking and navigating, toggling buttons, choosing
menu items.
You don't have to write any code at all to do that here,
so that's great.
So you can get started testing right away.
If you're doing an Android app, and you
haven't done any testing at all, there's
no excuse not to do this.
Because there's a free allotment of daily
tests that you can run in Test Lab.
So Test Lab is good to use before you publish.
But what do you do after you publish?
Well, many people are using Firebase Test Lab
in their continuous integration system.
So they can build and test, build
and test repeatedly, maybe with every check in,
maybe just nightly.
But after you publish, after you've finished that cycle,
it's too late.
So if there's crashes that you haven't found,
those could still reach your users.
The fact of the matter is it's difficult to predict and test
all these corner cases.
Your app may still crash in the hands of your users.
Now, Ibrahim, we found a really nasty bug in FriendlyPix,
right?
IBRAHIM ULUKAYA: Yes, Doug.
Actually, this was our first bug right after we shipped the app.
You know, we were happy, we shipped the app,
had some celebrations.
And we get this new fatal issue email.
And let's see what's going on there.
So we went to the Crashlytics console.
As we see, the crash-free statistics
went from 100% all the way to 70% in a matter of days.
So it was really scary for us.
But we went through, and we see that it
was in the first version.
We see how many people are affected.
And we went through, and we see that it was on the FPUser.
So we got some really good insight
to what's happening there.
Instead of like-- sometimes the [INAUDIBLE] code
may not give you the [? best ?] stacktrace.
But we have some clean stackrace here.
And by using this--
let's go back to our code-- then I
found out that I was always-- while I was building the app,
I was expecting the full name to be never null.
But the web client was sending null.
So sometimes we can't avoid everything.
And in this case, we've managed to talk with the web client
developer as well.
We fixed on both sides, and we made sure
that this issue will never occur again.
Since then, we were able to add a note here,
what happened here.
And we close the bug.
We shipped the new version.
And as you see, as soon as we shipped the version,
all crash-free users went all the way to 100%.
But we learned a fun lesson here.
And we also integrated the Slack.
And with the Slack integration, now,
when we have any alert like this,
we get an alert right in our Slack channel.
And all our developers from different clients
can jump in and talk to each other
and start the conversations again.
And also, we start using the user identified
in the Crashlytics, which you can set.
So in our crash reports, we can get some user ID.
This gives us another available information
that, if there's a data [? offending ?] us,
we can find it.
Maybe we can try to actually make up with this user.
Maybe we'll send a push notification
saying that, our app is much better now.
Do you want to try again, give us a chance?
DOUG STEVENSON: Yeah, I think it's
kind of funny that you used Crashlytics in the iOS app.
But we ended up finding a bug in the web app.
But it goes to show that sometimes it's
good to be paranoid about the data that's
coming into your app, even if it's coming
from a server you control.
Now, there's another problem that pretty much
all mobile software engineers have
struggled with at some point.
And that's race conditions.
Let's be honest.
Threading is really difficult to get right,
even for us at Firebase.
It's just not easy to predict how
all of your asynchronous and threaded work
are going to interact with each other.
We had a problem like this in FriendlyPix, didn't we?
IBRAHIM ULUKAYA: Indeed, yes, Doug.
And here, we get another fatal issue.
And this time, we had some load [? feed, ?] one
closure, another closure.
This was harder to replicate because this bug only
happened when there are multiple people actually interactive.
And in this case, let's go to our app
to give us some idea what's happening there.
So you see some feeds in my main view controller.
As I move to another view controller,
I actually-- since I'm using the real-time database,
I have those async listeners.
And now I want to disconnect those listeners
so they are not running on the back [INAUDIBLE]..
But if I want to go back to my original view controller,
I don't want to load the entire feed from scratch.
Instead, I just want to upload as I update and see
if those data is still live.
And if it's changed, I want to change it here as well.
And so, I'm creating some new listeners.
And I'm checking if those data are still there.
Let's say this index 0, my own post,
was deleted on the back end.
I came here, I asked a question, the async call returned saying
that that post was deleted.
Should I go and delete index 0?
That was the problem I had.
Because when I did that, I didn't realize
there were other async threads going on.
They actually added another feed to my feed--
another post to my feed, so my index may have already changed.
To find out and to prove this theorem,
I actually added some other custom keys,
like adding some other counts for different stages
of my code, using the custom keys on the Crashlytics.
And I was able to compare from some crash reports.
And at the end, we came to a solution
in our listenPost and then where we update the code.
Instead of relying on some index paths,
we have to find the [? root ?] index using the post ID.
And after we did that, we also started
integrating with PagerDuty.
Now we get pagers for the on-call developers.
And we have the Jira integration,
so we start tracing those bugs.
And last bit that--
earlier, they mentioned about the Crashlytics data
through the dashboard and with the BigQuery exports.
So we started using it as well.
By exporting our Crashlytics data to Data Studio
to be queried and using the template that already came out
from the Crashlytics, we have some really valuable data here.
We can see crash trends through different versions,
different devices, and through different keys and by files.
So we can see some trends.
And if there's some problematic data
over there or problematic file, we
can go through and maybe try to [INAUDIBLE] it.
DOUG STEVENSON: So obviously, crashes
are the biggest threat to the UX of your app.
And you can use BigQuery visualization to monitor
trends of crashes in your app.
But there's an even more insidious problem
that can happen.
And that's the overall performance of your app.
If your app is stuck loading something
or if it can't launch very quickly,
that's going to cause frustrations to your users.
And sometimes, I like to call that a soft crash.
Because if your app isn't doing what the user expects,
it might as well have just crashed
because the user's going to leave in frustration.
So when you develop your app, you're
probably building in your home or your office
with a great Wi-Fi connection and a fast connection
to the internet.
And you're probably using the latest hardware
for development, so all the latest devices,
in order to tighten up your test and debug cycles.
But this is not likely going to be the situation for your end
users, who might have poor connections on low-end devices.
It's really important to notice, if you
want to collect performance data for your app,
you need to do it from your user's point of view.
So whatever your users are experiencing,
you need to know what that is.
And we found some performance issues in FriendlyPix,
didn't we?
What did we discover?
IBRAHIM ULUKAYA: Yeah, actually, we
build our app usually using the Wi-Fi and the latest devices.
But sometimes, we can omit those issues,
we didn't see those issues while we were developing.
And using the performance metrics,
we can collect those data, and we can analyze it.
And here, let's go to our console.
As you can see here, I have some app start trace
and some network response latency
varying through different countries.
Let's see what's going on here.
The app start is a particularly important issue.
Because even if your app didn't crash,
if it is starting really late, some users
may assume that your app has crashed.
Or they think it is not responding,
and they won't use your app anymore.
So if you go through-- and I'll just go through some versions
and maybe explain some what are the improvements
we made so far.
And one of the issues was I was just
explaining-- we are loading all day sinkholes.
As much as we could, now, we are trying
to load them async from different [? areas. ?]
And at the end, we try to get signal together,
merge them together.
And the other things we found that--
I was not cropping the full-size images,
as I was letting people upload files as big as they want.
And that was creating some crazy download times
when our app was starting.
So we fixed that issue as well.
And going through that, now we see some different devices.
And in our version, we can see that we start with a really
slow app start time.
We work through it.
As we added some new features like blocking
users and other permissions, we kind of slow our app.
But we are always working [INAUDIBLE] to make it
as fast as it can be.
DOUG STEVENSON: Yeah, the performance monitoring
dashboard is very helpful to identify problems--
performance problems-- in your app.
And then also use it to verify that you actually fixed it.
So getting a before and after view is very helpful.
Now, you may have noticed from what Ibrahim showed you,
the dashboard is displaying data in aggregate.
So you're looking at averages, sometimes
among all of your users.
Aggregates are really good because it's showing you
all of the data from all of your users
or maybe just a slice of your users.
But you can't possibly process every single user's data
all at once.
But sometimes, aggregate data doesn't tell the whole story.
Sometimes you do need to deep dive
into a particular performance session.
Now, earlier today, you may have noticed that the Firebase
Performance team launched a whole new feature that
lets you browse and examine individual performance
sessions.
So you can deep dive into the performance characteristics
on a particular device for a particular user
at a particular period in time.
Now, Ibrahim, I know that you were able to give this
a close look before it shipped.
What did you discover in FriendlyPix?
IBRAHIM ULUKAYA: Actually, FriendlyPix
was one of the early adopters.
So we'll actually go to real data
we collected from a real app.
And this was a little bit-- bear with me,
we'll see how it works there.
So I'll dive into one of the sessions I have.
I will see in this particular case,
this is the app start time.
This is my feed view.
Some moment, even my feed view was loaded, my first view.
But we get stuck on the AuthPicker.
So something went bad [? when under ?] authentication
that it went 15 seconds.
So either something was wrong with the permissions
or the app that made, really, this user stall.
Let's look at another session.
Let's get the one we were seeing some info about it.
Here we go.
This is an iPhone 6 device.
It was a little older device, and I suspect something
different happened here.
As you see, it took more than two seconds
just for my feed view even start loading.
I suspect either my app was too big
or I was doing some animations and other stuff that
made the app really slow to even get loaded.
So using this kind of real data as some real metrics,
we can actually understand what's going on with our app.
And we can tailor our app so it can
work as fast as it can be through different devices
and in different conditions as well.
DOUG STEVENSON: So thank you, Ibrahim, for that close
look at everything that's new in Test Lab and Crashlytics
and Performance Monitoring.
And thank you all for joining us today.
I hope you have enjoyed the content today.
I hope you continue to enjoy the rest of the afternoon.
Now, Ibrahim and I will be around
and about the conference in workshops, [? #AskFirebase, ?]
other parts of the conference.
Feel free to flag us down and talk to us
about performance issues or anything else
that you come up with.
Thank you, and we'll see you later.
[MUSIC PLAYING]