TODD BURNER: Hi, everyone.
Thanks for joining me on the second installment
of our Firebase Test Lab series.
Last time, we went through the Getting Started flow.
In that video, we uploaded our debug APK to Test Lab,
and Test Lab ran our first robo test automatically.
As you saw in the last video, robo tests
navigate your app automatically, capture log files,
save a series of annotated screenshots,
and then generate a video from those screenshots to show you
the navigation performed.
You can use these to determine the root cause of your app's
crashes and they also help you find issues with your app's UI.
But what if we want to choose specific devices for testing
on, test for specific locales, or even set up auth?
No worries, we have a tool for that.
And you won't need to add any code your app to get started.
On this episode of Firecast, we'll
learn how to set up a custom robo test in Test Lab.
[MUSIC PLAYING]
Great.
Firebase Test Lab lets you have coverage
without much work on your end.
One example is robo tests.
They give you as much testing ability as possible
with minimal effort on your side.
In our first video, we saw a preview
of the power of robo tests.
And then this video, we're going to configure Test Lab to work
exactly how we want.
On the way, we'll learn about our test matrix and robo
configurations so you can set up your test
to meet exactly your needs.
So let's dive in and make a customized robo test.
Here I am in the testing dashboard.
As you can see, this is different from our first run
experience.
Now we give you the power to make
customizations you need to use Test Lab exactly how you want.
You can see my previous tests listed here,
and I can click in to see the results.
Up in the right corner, you can see this plus sign.
Click that and you'll see some options.
We can click right into robo test,
but for now I'm going to click Help Me Choose.
You can choose between the three types--
robo tests that automatically crawl your app
and find issues, instrumentation tests which
run Espresso, Robotium, and UIAutomator tests,
and game loops that can simulate actions of a player.
Today we're going to choose robo tests,
but each is useful for different situations
and you can read more in our docs.
After clicking Continue, we're presented with this upload APK
screen.
In Android Studio, I built my debug APK.
To do that, I ran a clean and assemble debug gradlew command.
And then I found the path of my APK by running this command.
Back in the console, I navigate to the path and upload the APK.
It's done uploading, and whoa!
I see a screen with many options.
We're going to go through this very thoroughly
so you know exactly how to get started when you try yourself.
So, I enter a robo testing dimensions page.
We test along different vectors--
specifically, device API pair, orientation, and locale.
Together, this builds a test matrix where
each combination is tested.
On the screen, I can choose from over 30 physical devices and 11
virtual devices with different versions of Android installed.
Further down, I can select Device Orientation
and which locales I want to be included in the test.
It's important to note the selection of orientation
and locale applies to each device and API
combo, which multiplies the number of tests for billing.
If you're on the Spark plan, your number of tests
are going to be limited.
On Spark, you can run five physical device tests per day
and 10 virtual device tests per day.
On our Blaze plan, you pay per testing minute.
Now let's open the advanced options.
Here, we have a couple options that are worth talking about.
To help you control your potential billing
exposure on the Blaze plan, you can start with a test timeout.
We recommend setting it at 120 seconds
and determining if it meets your needs based
on your app's complexity.
If your app uses Google Sign In for authentication,
robo will use its own account to automatically sign in
at the Google sign in dialogue.
Prior to the start of any test on a device,
Test Lab installs a special Google account on the device.
If robo sees a Log In With Google button,
it will click and then log into the app with that Google
account.
But if your app has a custom sign in scheme, such as email
and password, you can also tell robo
to enter that with the edit text fields
and log in successfully too.
You can do that here.
Finally, you can provide up to 3 deep links.
This means you can start your app
crawl from a specific location in your app.
Your app is first navigated normally from action start
and then each deep link is crawled for 30 seconds.
Back in the console, I can see the option
to save this template for future tests.
One really interesting and useful thing here
is the concept of the testing matrix.
These are the dimensions of our test matrix--
device, orientation, and locale.
Below the Confirm button says, how many tests will be run?
I click this and my tests start.
After a couple of minutes, I get this email alert showing me
the test was a success.
If I click the View Results link,
it leads me to a matrix specific results
page where I can see a summary of all the device, APIs
levels, and other options I selected back in the matrix.
Tests will show as passed, failed, inconclusive,
or skipped.
Here, all my tests passed.
In robo, if a test fails, it means
we found a crash in your app.
If I click into a test, I'll get the test
specific results like we saw in the first Test Lab video.
If you came from that video, this should all look familiar.
And that's how you use Test Lab robo tests.
Like this video and subscribe if you
want to hear more about Test Lab and our other Firebase tools.
Thanks, and I'll see you next time.
[MUSIC PLAYING]