>>Yvette Nameth: Last up today, we have Casey Burkhardt, who is going to talk about automated
accessibility testing for Android applications.
>>Casey Burkhardt: Thank you.
Good afternoon, everyone.
My name is Casey Burkhardt.
I'm a member of Google's Accessibility Engineering team in Mountain View, California.
I'm here today to talk about accessibility in terms of mobile apps, and specifically
how you can leverage some new tools to automate accessibility testing to a degree within your
own Android Apps.
Let's see what we have on the agenda today.
First we're going to cover in general what's accessibility, how does it manifest itself
on Android.
We're going to review the top developer mistakes that we see both at Google and in third-party
apps.
We're going to introduce a few tools and integrations of our tools to identify accessibility issues
automatically.
We're going to discuss how that tool works, and specifically what it will work for, what
it will not work for.
And, finally, we'll introduce how you can leverage this tool, these tools, in your applications
and in your tests to spot accessibility issues early and easily.
So first of all, what is accessibility?
Accessibility really, the core of it is helping users with disabilities access your products
and services, making sure it's very easy and straightforward for these users to use your
apps.
I like to phrase it a little differently than that, though.
I like to say it's about challenging the assumptions we make as developers about our users.
Okay?
So some examples of this.
When we're writing our apps, I'm sure everyone in this room who's written a mobile app before
or Web site or any sort of application has said, I can assume that the user can read
the screen.
Accessibility is about challenging that assumption.
But there's a number of other assumptions we want to challenge as well.
Can we assume that the user is able to actually interact with a touch screen?
Can we assume that the user's able to differentiate color?
Can we assume that the user is able to hear sound that my app is producing?
So accessibility is about taking all these assumptions and building software that doesn't
make them.
Basically, it's a component of universal design.
Accessibility also affects -- When we talk about accessibility, we often hear the phrase
"the long tail."
But, actually, we're talking about a substantial number of users.
One in five users in their lifetime will self-identify as having a disability.
So we're talking about a substantial number of people.
We're not talking about sort of fringe users or the long tail necessarily.
We're talking about a substantial slice of the population here.
The other thing I'd like to mention is, at one time or another, we all have a disability.
We have a functional disability.
So, for example, if you're driving your car, you can't be looking at your mobile -- well,
you shouldn't be looking at your mobile device.
If you're cooking and you have greasy hands, you may not necessarily be able to interact
with the touch screen on your device without getting it dirty.
So at one point or another, we all have some so far functional disability.
Finally, accessibility and testability are very closely related.
So, essentially, this is a technical artifact.
Most UIAutomation frameworks that modern-day platforms expose actually under the hood are
implemented by using the accessibility APIs on the platform.
So your UI tests are pretty much linked, they have the same behavior and the same code path
executes when you action something in a UI test, as will be executed by an accessibility
service or assistive technology on a given platform.
We had an example this morning where we were talking about offline maps in Google Maps
for Mobile, about two weeks spent debugging an issue which came down to a developer's
assumption, the assumption that the user could touch the physical display.
Whereas the testing tool, which most likely used the accessibility APIs, could not.
It's possible -- actually, it's probable -- that this is also -- was an accessibility issue
in this product.
For a user who's unable to touch the screen and is using another device as an input mechanism,
we can very easily see how they can run into the same situation.
And that assumption came back and actually produced a bug in the product.
So that's an overview of accessibility.
Let's briefly go over how this manifests itself on Android.
So what we see here are three accessibility services.
An accessibility services on the Android platform is sort of a long-running service that's meant
to be present whenever the device is being used or interacted with, basically, for the
duration that the device is on.
And accessibility services pretty much change the interaction model on the device.
So they allow the user to access content on the device in a way that works for them.
And it also allows the user to interact with that content or action controls on the device
in a way that they're able.
So the three examples I have here are the three accessibility services that Google ships.
The first is TalkBack.
It's a screen reader for Android that basically changes the interaction model to support a
series of gestures.
Users navigate using these gestures and move focus on the display and hear the content
that gains focus spoken to them through the device's text-to-speech engine.
BrailleBack is another service.
It works in a similar way, uses the same APIs as TalkBack.
But instead of producing speech through the text-to-speech engine, it will output device
content on a device like this one.
This is a refreshable Braille display.
It connects to Android phones and tablets over Bluetooth, and it allows the users to
essentially read dynamic Braille off this row of Braille cells.
And it's better in situations where privacy is important.
You don't necessarily want that device content spoken out loud to the world.
Finally, our third service is Switch Access.
It's intended to help users with mobility impairments access content on their device
by allowing them to interact with the device in a different way.
A lot of users with mobility impairments won't have the typical touch precision that we have
when we interact with a touch screen, so they'll use another device, like an adaptive switch
-- this is an example of one -- which basically provides a different input mechanism.
With this switch and Switch Access enabled on an Android device, you can essentially
control the entire device just through these two switches.
In addition to accessibility services, the platform exposes a couple of core accessibility
features, too.
There are affordances on the platform for manipulating the size of text as well as supporting
magnification of the entire display.
There's a mechanism to invert colors.
So people who are light-sensitive, like myself, can view apps in a more comfortable manner.
There's affordances for changing the appearance of text with the aim to increase contrast.
There's also support for captioning.
So across the platform, video views and other apps that prevent video content have a way
of discovering the user's captioning preferences.
And there's also features that basically allow for color correction.
So users who are colorblind and unable to distinguish two colors can essentially shift
the display's color space into areas that they're more easily able to distinguish.
So we've covered a lot of accessibility features that the platform offers, a lot of accessibility
services that are available.
And it's important to think of accessibility not only in terms of one of these services,
but all of these services, and even combinations of the services at the same time.
I still hear people come up to me and they go, oh, I tested my app with the screen reader.
It worked great.
All the announcements were -- came through at the exact time I would expect.
So I'm done with accessibility.
Surprise.
You're not.
You should be looking at all of these services and make sure that your app is exposing its
content semantically in a way that works well with each of these features.
So some common developer mistakes that we see.
Probably the most common is mislabeled or unlabeled content.
So, for example, if you use an image view somewhere in your Android application and
that has some meaning to the user, you need to provide a content description to allow
a screen reader or a Braille user to understand what that image represents.
Equally important is making sure that your description is reasonable and you're using
the right attributes to expose that information.
We want to ensure that we have large enough touch targets.
So users, for example, without fine motor control may not be able to hit smaller touch
targets on the display.
So we recommend that all your touch targets are at least 48 by 48 DIPs.
Ensure that you have proper foreground-to-background contrast.
Essentially, what you're looking for here is a ratio of four and a half to one between
the background and foreground color that your apps use.
If you're not sure how to compute that, there's tons of online tools available to help you.
And, finally, you want to make sure that your view is exposing the right semantics.
If your view is clickable, advertise it as clickable.
Make sure you're using the affordances in the platform for handling clicks.
If it's not clickable, make sure it's not reported as clickable.
This, for example, can prevent users from efficiently navigating your app with Switch
Access.
So these are a few examples of common developer mistakes.
But how can we find of them?
How do we discover them?
And I think the best answer to this is, through manual testing.
Manual QA involves a human going through your app with assistive technology enabled and
assessing its -- the user experience for correctness.
This, in my opinion, is the only way you can completely get a perspective for how users
with disabilities are going to be interacting with your app.
But manual testing is a costly and time-consuming process.
So our team set out to think about ways in which we can improve this and speed up manual
testing and reduce the number of developer cycles and back and forth that have to occur
with QA organizations.
So we want to find accessibility issues really early on.
And we can do that for a lot of the common issues that we run into.
In that vein, we created what we're calling the Android Accessibility Test Framework.
It is essentially a Java library that is written and includes logic to detect a number of accessibility
issues using Android UI constructs.
So we can look at the views presented within your application.
Or we can look at another representation of them called accessibility node info.
Anyone who has used UI automation on Android is probably familiar with that.
And we basically have implemented this one core library that contains all of the testing
logic.
We made it very simple to add additional checks.
So if you would like to contribute or give us ideas on how to expand this library to
find more accessibility issues, feel free to contribute to patch or get in touch with
us.
We've also made it really easy to integrate this framework into other testing frameworks,
which is what -- which is our current approach.
So what we have done is basically taken our accessibility test framework and included
it inside Espresso and Robolectric as an optional component.
So you can get accessibility test coverage that relies on the coverage of your existing
tests.
So let's see what that actually looks like.
So for Espresso, you would enable this feature by, first of all, getting the dependency from
the Espresso contrib directory and somewhere in your test life cycle call accessibilitychecks.enable.
For the lifetime of that test, we will run our accessibility evaluations on any view
that you interact with through an Espresso view action.
So we'll look -- if you click a button with a view action, we'll look at that button and
that button, potentially the UI around it, for accessibility issues.
If your touch target is too small, if you are missing a screen reader description, we
will actually go and failure existing test.
So with very little work, you can take advantage of this integration in Espresso.
And also of note, you can customize the behavior if you'd like and potentially exclude known
bugs with a class known as accessibility validator.
Here's what an example failure looks like.
In this case, this is the actual Espresso output.
It identifies that there's an accessibility error.
It gives you the view I.D. and Espresso's typical view representation and notes the
actual particular error.
In this case, there's a speakable description missing for a screen reader.
We've taken a similar approach in Robolectric.
The setup is slightly different.
Instead of calling a static method, you simply annotate your test methods or your relevant
test classes with the accessibility checks annotation that will automatically enable
accessibility evaluations.
And any view you interact with, with Robolectric's shadowview.clickon will receive an accessibility
evaluation.
It's important that you have don't call view.performclick directly or invoke click listener logic because
that will potentially bypass our integration with Robolectric.
We've got more tools for suppressing known issues and other advanced functionality coming
soon.
In fact, I think our poll request for some new things in Robolectric just landed this
morning.
So please check that out.
Here's of what an example of a Robolectric failure actually looks like.
Again, you see the message that view has accessibility issues and missing speakable text for a screen
reader.
You get the actual stack trace, so you will see the call to click on that resulted in
our evaluation failing.
So what can't we do?
So, first of all, our coverage is only as good as your coverage.
If your tests don't cover your application's UI in a complete way, you may not get accessibility
evaluation coverage for that UI either.
So we rely on your existing tests to decide how we work and what we do.
Secondly, we err on the side of caution.
If we're not sure it's an accessibility issue but it could be, we emit a warning either
to the test output or logcat depending on what platform -- what integration you are
using.
So please check the logcat or the test output for additional potential accessibility warnings
as well.
And, finally, we are not a replacement for manual accessibility testing.
We still very highly advocate the use of manual qualitative QA to go through your app with
accessibility services and understand the experience.
The example I like to use here to illustrate this is we can tell if a particular control
has a description for a screen reader or not.
But we can't tell if that description makes sense to the user.
It's always good to have manual QA go and review your implementations.
You can use the automated test framework as a mechanism to speed up that step.
So what's next for you?
Go and try this out.
If you have Espresso or Robolectric tests, use the accessibility checks class or annotation
to turn this feature on and see what you get.
You can even -- you probably should expect a large number of issues.
We find that most apps that turn all the checks on at once run into issues that they then
have to go through and suppress.
But that's a good thing.
You are finding a lot of issues in your apps.
And you're improving the accessibility overall of the ecosystem, which we really appreciate.
If you have any questions about the project, something doesn't seem to be working as you'd
expect, please reach out to us on GitHub.
File an issue on our GitHub page.
If you have ideas for new accessibility tests we can add to our suite, please also let us
know.
And also part of the reason we're here is to find what's next and learn about the next
platform we want to integrate with.
If you have thoughts or recommendations or suggestions for any of that, please reach
out to us as well.
Thank you.
[ Applause ] >>Yvette Nameth: Thank you, Casey.
We will take a question.
Have you considered including accessibility evaluation as part of the Google Play upload
process?
>>Casey Burkhardt: That's an excellent idea.
We have nothing to announce today regarding that.
But it's definitely something our team has been discussing, and we'd like to pursue it
further.
Yeah.
>>Yvette Nameth: Well, that was short, so I will give you one more.
Are there plans to support accessibility automation integration with Google's UI Automator V2?
>>Casey Burkhardt: It's something we are definitely looking at.
So UI Automator today uses a lot of the same constructs that we can evaluate.
So it's definitely one path to move forward and expand the suite of frameworks that we
actually integrate with.
But we'll be talking to the Android team about that.
Definitely.
>>Yvette Nameth: And thank you.
>>Casey Burkhardt: Great.