TODD BURNER: Working on mobile teams
at large and small companies, I've
noticed it's often tricky to figure out good strategy
when it comes to testing.
With limited resources, how do you
know which devices to test on, and how do you
do a consistent job every time?
When new users come into your app,
you need to make a good impression
or they won't come back.
And if you aren't sure about your app's behavior
on different devices before each release,
you're going to run into serious surprises in your crash reports
after you launch.
That's why we built Firebase Test Lab.
It helps improve the quality of your user's experience
by letting you run tests on real devices in our data centers
before you ship.
It's great for small apps with a bunch of functionality
you can use without adding a single line of code.
And for larger apps, Test Lab offers a ton of customizations.
So let's get started with Firebase Test Lab.
Welcome to another episode of Firecast.
I'm Todd Burner from the Firebase team.
Firebase Test Lab lets you test your app
on real and virtual devices taking
advantage of simple pricing.
You can test any app whether or not it uses Firebase
and take advantage of Test Lab.
So here I am in the Firebase console.
If you've never used Test Lab on this Firebase project,
you'll be presented with this first-run experience.
The next time you visit, you'll see a more customizable set
of options.
If I click Test Lab in the project,
I'm presented with this screen asking me to upload my APK.
When I do, it'll perform a free automated test of my app.
This is a special sort of test called a robo
test where Test Lab will crawl my app in an intelligent way.
This test doesn't require any code or configuration upfront.
Let's talk about what type of APKs
you can upload to Test Lab.
Debug APKs are OK as they don't need to be formally signed.
This is the easiest way to get started with Test Lab.
On the other hand, incremental builds with Instant Run
will not work because everything we need is not in a single APK.
So here I am in Android Studio.
I usually do a full clean build from the command line.
So I run a gradle clean and assemble debug command.
I'll run that here in the Android Studio command line.
While that builds, I should mention
it can be tricky to find the debug APK that you built.
One tip is that the output of this command
will yield the output location of the debug APK.
If it's your preference on Mac, you
can then CD into its directory and type
open dot to open a finder window, which
makes it super easy to drag that APK into the Firebase console.
I'll do that here.
Now I drop my APK into this field.
After it uploads, the first test automatically
starts running on a real Google Pixel device at one
of Google's data centers.
It could take a while for Test Lab
to finish its automated test.
You can wait for it to complete in the dashboard,
or you can come back later after you get an email update.
When those tests are done, I'll get a convenient email alert
that lets me know which test was completed
and whether or not it resulted in any crashes.
In fact, anyone with the reader role or more in the project
would get this email.
I can then click into the dashboard to see more.
The dashboard shows me the path of where the robo test went
and my app, how long it crawled, and screenshots
of the unique screens robo found.
If I click in the Logs tab, I can
see the full log cat output of the app
while it was being tested.
The Screenshots tab shows the individual screens
that my app displayed and highlights
some of the interactions between my app and the robo tests.
Finally, we have the video and performance screens,
which will show some of the performance metrics collected
while my app was being tested.
Like I mentioned before, you can run tests
against pretty much any APK.
This means debugger release build types
with or without Firebase integrated.
And you can test multiple different apps
all in the same project.
So let's talk about pricing.
On the Spark plan, Firebase Test Lab
is free for 10 virtual tests and five real device tests per day.
If you upgrade to the Blaze plan, you'll be billed $5 US
per physical device hour and $1 per virtual device hour,
but you only pay for when your tests are actually being run,
not device allocation, prep, or teardown.
This means that if your tests run on five different device
configurations and you've submitted a test that
takes 12 minutes to run on each device,
that's a total of one hour, which will cost $5.
Or if you have six different device configurations
that each run for 10 minutes, that's
also a total of one hour for $5.
The math is pretty easy, so you get the idea.
You can choose the maximum amount of time per crawl
to make sure that you stay under budget.
Oh, and there's one other thing.
If you don't want to pay for testing,
you can still take advantage of Firebase Test Lab
by uploading your APK to an alpha or beta channel
in the Google Play Developer console.
All you have to do is opt in to the pre-launch report
on Google Play.
From that point on, every APK you upload to those channels
will receive a five-minute robo test
across a fixed set of devices at no charge to you.
After a while, those test results
will appear alongside your APK in the channel
where you uploaded it.
Congrats!
You just had your first robo test in Test Lab.
Subscribe and watch the next video,
where I'll talk about setting up your first advanced tests using
specific devices and Google login.
If you have any questions, reach out to us
on Twitter with the hashtag #AskFirebase.
or in the comments below.
I'm Todd Burner.
Thanks for watching, and don't forget--
test early and often with Firebase.
[MUSIC PLAYING]