[MUSIC PLAYING]
QASID SADIQ: Hey, guys.
My name is Qasid Sadiq and I'm an engineer on the Android
Accessibility team.
So getting right into it, if you use standard widgets,
like the ones that we provide for you in the platform
AndroidX and material, you're going
to have accessibility built in out of the box.
And you're going to have a pretty easy time doing
accessibility work, mostly doing things like adding labels.
But if you have to write your own custom
widgets or your own UI toolkit, or anywhere in between,
that story is going to get a little bit more complicated.
You're going to need to do quite a bit
of the accessibility heavy lifting.
So to understand why, I'm going to walk you
through one of the widgets that we worked on internally,
in terms of accessibility.
And, through this, you'll get to understand a little bit more
about the accessibility system.
You'll get to understand the challenges
that you're going to face when you're doing this sort of work.
And you'll also get to understand
how to scope accessibility work beforehand, because that can
be a little bit unpredictable.
Now, before we get into that story,
let's talk about accessibility at a high level real quick.
So there's your user and that's your UI.
Your UI presents information to the user, information
like the content that they want, and information
on how to act on that UI.
The user takes that and they act on your UI
to make it do things.
And that's a complete user experience.
If any part of that is broken, even subtly,
then your application is broken, as far as a user is concerned.
Now, that seems simple enough.
But when you take into account accessibility and all
the dramatically different kinds of accessibility users
and the way that they interact with the world,
things get a lot more complicated, right?
Because we have all these accessibilities and we
have variances in the accessibilities.
We've also got combinations of these accessibilities.
So we really don't expect you, as app developers,
to figure out each and every single one of these users,
make profiles for them, and build a separate interface
for each of these users.
That'd be overwhelming and a pretty good way
of creating an awful accessibility
experience across applications.
Instead, what we have is we have the concept
of an accessibility service.
Now, an accessibility service is an application,
which takes the UI presented and adjusts it
for a particular kind of user.
So, for example, TalkBack is an accessibility service.
It takes information presented visually and speaks out
loud to a user with visual impairments.
Similarly, Switch Access allows a user
with mobility impairments to drive the whole UI
with a series of switches, even as little as one.
Now, the way an accessibility does this is it
consumes a generic representation
of the information presented on screen
through the accessibility framework.
So your task, as an app developer,
becomes pretty simple--
or simpler.
Make sure that every single thing that your application
presents to the user is expressed
to the accessibility service.
And make sure the accessibility service
can act in your application or your UI any way
that a user can.
So let's actually go a little bit more in-depth,
in terms of this interface between an application
and a user.
And we're going to break into the lower levels of this.
There's three core objects to this.
The first one is an accessibility node info.
Accessibility node info represents
what's currently on screen.
Each accessibility node info typically, but not always--
there's a few notable exceptions-- corresponds
to a view.
It captures information like its position on screen, the text,
the content description associated with it,
what you can do to the view, and a bunch of other stuff
related to state and other things.
You can see that, once we take into account parent and child
relationships, you'll have a node tree similar to a view
hierarchy.
And you can see that the accessibility service
through this will have a pretty good understanding of all
of the information presented on screen.
So now that the accessibility service
can query for what's on screen, it needs to know when to query.
And that comes from an accessibility event.
This is something that's sent from the application
to the accessibility service whenever a change happens
within your UI.
Let me walk you through an example.
Let's look at this view hierarchy on the left
and the node tree on the right.
And let's focus on that green highlighted view.
So if something changes in that view,
like if some view is removed, an accessibility event
is sent from the highlighted view.
The accessibility service knows to invalidate
that part of the node tree hierarchy
and query for a new one.
And now it has the updated representation
of what information is being presented to the user.
Now, at any given moment, the accessibility service
has an accurate representation of what's on screen.
But what's missing, at this point,
is the ability for the accessibility service
to act on your UI, typically on behalf of the user.
That's captured in an accessibility action, which
is an object which effectively maps an action ID, which
could be something like a standard action,
like scrolling, or clicking, or long pressing, to a bit of code
that you want executed when something like that happens.
So with these core pieces, we can
start talking about the main story, which
is ViewPager2, which is a widget we're
going to talk through today.
So several months ago, a couple of my colleagues,
Jel and Jacob, decided that they're
going to make a new version of ViewPager, one not tied
to the issues that ViewPager1 is stuck to, partially
because they're going to utilize the power and the flexibility
of RecyclerView.
And they did the right thing and, very early
on in the development process, they
approach the accessibility team, which is me and my colleagues,
and they asked us, hey, can you guys
help us to accessibility for this?
And one of my colleague, Sally, said, yeah, of course I can.
And she jumped right into this.
This is the work that she did.
So before anything else, we need to understand
what a ViewPager is.
It's effectively a view that allows you to flip
through a series of pages.
These pages can be views themselves,
hierarchies, fragments, whatever you expect.
And because the way the core of this functionality
is allowing you the user to flip through a page,
we can start there, in terms of accessibility.
We want to expose the ability for a user to go from one page
to another to the accessibility service.
If you remember, that can be done through something
like an accessibility action, which captures user actions.
So we're going to use the high level API, viewcompat.repla
ceaccessibilityaction.
And we're going to take--
override the standard behavior for the action,
accessibility scroll backward.
And we're going to override it for this particular view.
So whenever the user asks its accessibility service
to scroll something backward on this view,
this code over here or this lambda
is going to be called, which fundamentally
documents the page count.
Set current item, sets the current page.
Now, we can do for actions scroll forward also.
Let's throw this into a function update page
accessibility actions.
And we'll call this on initialize or something,
so this exists on the ViewPager.
Now, the accessibility node info associated with this ViewPager
has these two custom actions on it,
accessibility scroll backward and accessibility scroll
forward.
That's great.
But, if you flip to the first page,
action scroll backward is still there.
This is a bit of a lie, because a user can't scroll backward
and neither should the service be able to either.
So we're going to add a condition to this.
Let's not add an action scroll backward on the first page.
Let's not add an access scroll forward on the last page.
And because our state of actions is not
dependent on the state of the view or the item,
let's clear out any of these actions
before we add them again, so we don't
leave anything stale on it.
And let's call update page accessibility actions
on set current item, because that's a dependent state.
Now, the actions are correctly reflected
based on the page number.
Now, ViewPager also has something--
also has a vertical orientation or a concept of orientation.
And this brings up an interesting point.
Although, technically, action scroll forward
and scroll backward still allow you
to use a ViewPager in a vertical fashion,
and it makes complete sense, there's
still something missing here, which
is a concept of absolute direction.
Certain accessibility services care a lot about this.
Like, someone driving their UI through their voice
may say scroll up or scroll down.
And this is something that's obviously
presented to the user in a very subtle way.
So we have to present it to the accessibility service.
So we're going to add some actions, which
allow the user to flip through pages in an absolute direction.
And that comes in action page up and down or left and right,
depending on the orientation.
And, again, we do the same thing.
We invalidate the actions beforehand,
and we update the actions whenever
the relevant state changes.
So now we not only have logical actions,
but we also have absolute direction actions.
And we have it for the horizontal orientation also.
Now, this talk about logical direction
versus absolute direction may get
you thinking about another case, which is right to left UI.
So, over here, Urdu is being displayed,
which is a language that's laid out right to left.
And the first page is on the far right.
So for this, again, we do something similar.
Based on whether it's right to left,
we flip action page right and action page left.
And, again, we go through this whole cycle of invalidating
the actions, updating state.
And you get it.
So now, regardless of localization,
our actions are added correctly.
Now, at this point, our user is able to do--
or our accessibility service is able to do anything
to this ViewPager that our user can do to it.
But this ViewPager is also presenting information.
In this particular implementation,
it's indicating stuff like a page count.
And that was missing to the accessibility service.
So the way we're going to do this is we're
going to break up--
open up the API.
And we're going to mess with the relationship
between the view and the accessibility node info.
We do that by overriding on initialize accessibility node
info on the view.
And the particular thing that we care about is collection info.
Now, collection info is an object
on accessibility node info that helps
represent view groups with a uniform set of views
below them.
So, for example, a list view has a collection info--
certain RecyclerViews have collection infos,
Tab Layouts do, and now our ViewPager
is going to have that, too.
We're going to get the row count, column count,
throw that into the collection info.
And we're going to associate that with the accessibility
node info.
Now, you may be wondering why everything over here
is accessibility node info compat,
and why we're wrapping the original accessibility
node info.
Accessibility node info itself is an object in the platform.
Accessibility node info compat is an object in AndroidX.
Now, we wrap-- we ask app developers to wrap
their node infos in node info compats,
because that allows us to backport API back to API 19,
and also allows us to backport certain bug fixes back
to API 19.
So, in general, this is just good practice,
even if you're not utilizing a new API that you
can't use on an older device.
OK, so that's great.
We're, for the most part, representing our ViewPager
pretty well to the accessibility service.
But with a little bit of testing,
we find that that page count starts to go stale.
And to understand where this bug came from,
you kind of have to understand how ViewPager really works.
See, ViewPager doesn't manage its items itself.
And instead of the SubView of a RecyclerView,
which manages its items.
And whenever a change happens to the ViewPager,
the RecyclerView sends the accessibility events.
When the accessibility service receives those events,
it invalidates that part of the hierarchy
and re-queries for it.
Notice how that ViewPager was never queried for again.
So now all of the information associated with it
is stale, including that column count that
was surfacing the page count.
So what we can do in this situation
is we made sure that the accessibility events
for the changes that we cared about
were sent from the ViewPager, instead of the RecyclerView.
Now our ViewPager not only allows accessibility service
to act on it in every way that a user can,
but it's presenting all of the information
to the accessibility service that it presents to the user.
And some of that was done through
the accessibility framework.
I mentioned the current index.
That was already done through the accessibility framework,
based on code that was used in RecyclerView.
So that was probably a good call by Jel and Jacob
to use RecyclerView here.
But it sounds great, but you guys
may have noticed that a lot of these fixes involved
were a little bit more complicated and a little bit
harder to come across, if you don't
have an intimate knowledge of accessibility.
And, in fact, some of these things,
they don't seem as straightforward as you think.
Like, for example, you look at accessibility scroll
forward and accessibility page right.
Why isn't it page forward?
Or why isn't it scroll right?
Why can't we do something that seems
a little bit more consistent?
And some of this has to do with the fact
that this is Legacy Code and we're
working with certain older accessibility services that
need to be supported, right?
Some of it has to do with the fact
that this API is pretty complicated.
So because of all of this nuance, and age,
and complexity, and the lower level accessibility APIs,
the more you try to work here, the more difficult
your life is going to be.
The more you're going to really need
to understand accessibility, even beyond what
the documentation presents.
This is captured in this chart over here.
This is our observation of how much accessibility
work you're going to need to do, based on how custom your UI is.
These are our observations from inside Google.
So at the easiest end-- what we hope everyone does, but really,
realistically, is not possible--
is that you use all the standard widgets
we provide to you in our frameworks.
These come with accessibility built in out of the box,
and all you have to do is label your images,
which shouldn't take long at all.
Now, once you move into a new widget territory,
you can see this is where things start getting more non-trivial,
like we just saw with ViewPager, which
was a reasonably well-built widget which utilized
some other standard widgets.
But things were kind of complicated.
And if you have to start building a new UI
toolkit, or a new UI platform, it's
going to be very, very complicated.
You're going to need multiple engineers working full time
on accessibility to make sure you don't
hurt your accessibility users.
So what we encourage you to do is move to far, to the left,
of this chart as possible.
And if you really have to be on the right side of this chart,
then make sure you are honest about how much
work it's going to take, and you allocate those resources.
Because, if you don't, you're choosing
to hurt your users in this way.
Now, the last bit that I want to touch upon,
which is kind of important, is testing.
It's surprisingly easy to ship bugs with accessibility,
especially because accessibility is not a used case or code
part that's exercised often.
So there's a couple obvious ways of testing.
The first is just open up the accessibility services
yourself.
Run TalkBack, run Switch Access, see how it works.
Typically, this is a good way of finding those very subtle bugs
that don't seem that significant when you're writing code,
but turn out to be very frustrating for users.
The other is unit testing or automation testing.
These take typically two forms.
One is you perform an action on a view,
and you check the state on the view.
And the other is you create some state on a view,
and you create an accessibility node info, based on that view,
and you check to see that the node info state reflects
that view correctly.
So the two really big key takeaways here--
if you guys didn't pay attention to any other part of this,
the two things that you can't forget about are, first,
everything that you express to your user,
you have to express to the accessibility service.
If you don't, then what you're saying
is you don't want the accessibility user
to be able to consume that UI or act on your UI in that way.
And the second is your accessibility work scales
dramatically with the amount of custom UI you have.
So, again, try to use standard widgets as much as possible.
Try to avoid using custom UI as much as possible,
so you can really focus on the value
that your application or your UI delivers to the world.
And if you really have to write your own UI,
because there are warranted situations for that--
or your own custom widgets--
then just be very honest about how much accessibility
work it's going to take, and scope it in advance,
and dedicate those resources.
Otherwise, you're going to catch yourself in a situation
where you're about to ship, and you have an inaccessible
product, and it's going to look pretty awful,
and you're going to hurt you users pretty badly.
Either way, thank you guys for your time.
I really appreciate it.
[MUSIC PLAYING]