[MUSIC PLAYING]
CAREN CHANG: Hey, Everybody.
I'm Caren, Engineer on Android Developer Relations team.
And today, I want to spend some time talking
about accessibility and Compose.
So we're going to start by talking
about two different ways to think about accessibility
for your app, and how Android framework enables
apps to be more accessible for a wide range of users.
Then building off of that, we're going
to talk about how Compose was built with accessibility
in mind from the start, and how you
can make your Compose apps more accessible.
Finally, we'll end with a couple of best practice tips
that you can keep in mind in the future
as you're developing apps.
All right.
So to start off, I wanted to give a quick overview
on the different ways to think about accessibility.
The first aspect to consider is the design of the app.
So this means making sure that all the elements in your app
are easily visible and interactive.
And this is important because it makes
it more accessible for the all users in your app.
Another aspect to consider when thinking about accessibility
is whether your app works well with the different assistive
technologies.
So sometimes, we forget that not everybody interacts
with the device the same way.
And sometimes, users may be using devices like keyboards,
or services like screen readers to help them
interact with their device.
So when considering accessibility for your app,
it's important to also consider whether you're
app works well with these different types of services
and devices.
So today's talk, we're mostly going
to be focusing on this as a technology slide,
so that we can build a better understanding of how
our Android apps interact with accessibility services
and how you can make your apps to be
more accessible for the wide range of users your app has.
All right, so the best way to start
thinking about supporting the variety of users in your app
is by looking at the different ways a user can
interact with an app.
So here we have a simple app with a Checkout button.
And we're going to look at the different ways a user can
interact with this Checkout button.
The first and most common way that I
think we're all familiar with is by tapping on the Checkout
button.
However, sometimes, a user may be
using TalkBack to interact with their device,
and therefore, your app.
So TalkBack is a screen reader that
helps users with vision impairments
navigate their device.
Using TalkBack, users can interact with the device using
swipes and gestures, and hear spoken announcements
about what's displayed on the screen.
So if a user was using TalkBack to interact with our app here,
they would navigate to the Checkout button with swipes,
and then they would hear the announcement Checkout button,
double tap to activate, and then they could double
tap anywhere on the screen to activate the Checkout button.
Voice Access is another accessibility service
that some users use to help them access their device.
Voice Access allows users to control their device
with spoken commands.
So if a user was using Voice Access to interact with our app
here, they would give the voice command, tap Checkout,
and similarly, that would activate the Checkout button.
So here, we've seen three different ways
to interact with the Checkout button here.
But of course, there are many other different ways
that the user may be interacting with this app
based on which accessibility service they're using.
So the question now becomes as an Android developer,
we only built our sample app in one way,
and we didn't really think about how
it would interact with these different accessibility
services.
So how does that actually work?
Well, most of the work starts in Android framework.
So what Android framework does is it takes a look at your app
and it translates it in a way that
allows different accessibility services to adapt your app
based on the user's needs.
In order to do this, what it does
is it creates a tree of nodes that reflects what's currently
displayed on the screen.
And each of these nodes is referred to
as an AccessibilityNodeInfo.
So AccessibilityNodeInfo contains
a lot of details about that specific node,
but I want to highlight two of the most important pieces
of information it contains.
The first is the label, so for example,
the content description that you set on an image button
to describe the purpose of the button.
The label is really important, because it
allows users to understand the purpose
of each element in your app.
The second is a set of actions that are available on elements.
So for example, for every element,
we need to know whether there are actions like tap,
or long click, or if it's some kind of text view,
is there an edit action available?
The actions allow accessibility services
to know how the user can interact with the element.
And you probably have never needed
to think about AccessibilityNodeInfo,
or the properties contained inside it,
mostly because these properties are
filled in automatically for you if you're
using built in components.
So for example, the framework automatically
populates the label text field for text elements
and certain actions for specific widgets like the switches
or radio buttons.
And so it's with all this information
that then allows TalkBack to make announcements
based on the element's label and available actions.
And similarly, because of all this info
that AccessibilityNodeInfo contains,
accessibility services are then able to navigate and interact
with Apps.
An Android framework creates this tree
of AccessibilityNodeInfos because it
wants to help developers represent
their app for accessibility services,
but also, you as a developer have the capability
to access the tree to create accessibility tools to support
more users.
All right.
So now that we have a better understanding of how
the framework works with your app
to enable different accessible services,
we're going to dive into how this all translates in Compose.
We're going to start at the beginning at what
happens when you create composable functions.
So when you create composable functions,
and the composable is evoked, a composition
is created to describe your UI.
As part of the composition, a semantics tree
is also generated.
So this semantics study is especially important for us
in this talk, because it contains all the information
about the meaning of each node.
And some of the information it contains
includes the label of each element,
the actions that each element has, or what kind of component
it is.
And similar to the old view system where
AccessiblyNodeInfos were automatically populated
for you, if you're using built in components,
the semantics tree is also automatically generated
if you're using components from the Compose Foundation library
or the Material library.
And so it's with the semantics tree
that the AccessibilityNodeInfo tree is then
able to be generated that is then able to power
accessibility services.
What this means is that composables with no meaning
are not included in the semantics tree, which
is then not seen by any accessibility services.
So here's a more concrete example
where we have a composable with a column.
The composable includes a button, an image, and a box.
So the button has an on click action associated with it.
So it has some kind of meaning.
So it's included in the semantics tree.
The box has a content description,
or a label included with it.
That means it also has some kind of meaning,
so it's also included in the semantics tree.
However, the image, as you see, is only
associated with the drawable.
That means it has no meaning in terms of actions or labels,
and therefore, it's not included in the semantics tree,
and therefore, not seen by accessibility services at all.
And the reason that Compose doesn't include elements
with no semantics is because omitting these type of elements
with no meaning improves the navigation for users
of accessibility services.
This allows them to more easily navigate through your app
by interacting only with the meaningful elements.
What this means is that sometimes,
Compose finds a need to merge nodes.
So sometimes, Compose merges nodes
by grouping certain content together
to enable easier navigation.
And this is automatically applied
for things inside a list.
So any items inside a list are automatically merged,
and also any elements inside a composable
with a clickable modifier are also automatically merged.
Sometimes, you might find the need
to specify your own merging behavior for a composable.
So a more concrete example, as we see here,
is an element with a drawable and some text.
So if we don't specify our own merging behavior
in this example, accessibility services
will see these three elements separately
as image and the two pieces of text.
However, because it might make sense
to group all this content together,
something we might consider doing
is to set the merge descendants property to true.
That will allow accessibility services
to see this element as a whole instead.
And similar to what we talked about
before, another way to achieve the same result
is by adding a clickable modifier to this whole element
row in order to merge it all together.
But something to keep in mind is that if you're
specifying merging behavior, descendants
that are set for merging are not merged with its parents, which
might be kind of hard to parse.
So let's take a look at an example.
So here, we have a clickable button,
the bookmark inside a card, which also
has its own clickable action.
As we mentioned before, if you apply a clickable modifier
to an item, it automatically merges them.
But here, since the Bookmark button has its own click
action, it doesn't want to be merged with the card.
That would get rid of the click action of the Bookmark.
So here, the Bookmark one preserves its own click action,
and the card also preserves its own click action.
Something else to consider when you're
specifying merging behavior is semantics properties.
The first is that semantics are additive.
And that means that, when you group together composables,
all their semantics are combined.
So here is a more concrete example.
This is the one that we looked at last time
where we grouped all the elements together
because they made sense to be grouped together
with the clickable modifier.
So something we might consider doing
to take it a step further is then
add a content description to the image.
When we add the content description to the image,
it will be combined with labels of the text
because it's all part of the row.
If you can't achieve the desired behavior
just by adding semantics properties,
sometimes, you might need to use the clearAndSetSemantics method
instead.
So going back to our last example, after we merge it,
we might find the need to specify
the order in which the content description is read aloud.
So for that specific use case, we
could use the clearAndSetSemantics method
to further specify how we want the content
description to be laid out.
And if you ever want to take a closer
look at the semantics tree for a composable,
you can always use the Layout Inspector in Android Studio.
And for each node in the semantics tree,
the Layout Inspector will show you the specific semantics
set on that node.
Similarly, you can also use the printed log method
and your Compose Test to log the full semantics tree
and see it printed out.
But the important thing to remember here
is that if you're using components,
they support accessibility out of the box,
so you hopefully don't have to worry about most of this.
If you're building your own custom components, however,
such as your own radio button or your own checkboxes,
that's when you want to pay closer attention
to accessibility support.
And a good way to do that is to check out the material
components implementation to better understand
how to utilize the Compose Foundation library that does
have the accessibility support to build off of that.
And a more concrete example is, for example,
if you wanted to build your own text field, a good way to start
would be looking at how the text field is built in material 3
based on the basic text field from the Compose foundation
library.
All right.
So to wrap it all up, we're going
to end with a few general accessibility best practice
tips to keep in mind for the future.
The first is to design for accessibility.
So if you consider accessibility from the start,
this builds solid foundation for your app
to support accessibility in the long run.
Next, always remember to label your elements.
Having concise and precise labels
helps all your users understand the purpose of each element.
And finally, if you can, always try to manually test.
Testing your app with accessibility service
such as TalkBack helps you understand
the areas for improvement.
As always, we're listening to your feedback.
So if you have any suggestions about how
to improve Compose and Accessibility,
feel free to file an issue.
And that's it for me.
Thanks for listening to this talk,
and thanks for interest in Accessibility.
[APPLAUSE]
[MUSIC PLAYING]