CAREN CHANG: Hello.
I'm Caren, an engineer on the Android Developer Relations
Team.
In this talk, my colleague, Anastasia, and I
will tell you all about the new developments
and accessibility for Android developers.
We'll touch upon improvements in Android Studio, new APIs,
and advances in Jetpack Compose.
It's important to remember that accessibility is
an important part of any app.
And by integrating accessibility features and services,
you can improve your app's usability for all users.
Let's start by looking at improvements in developer tools
that will help you build more accessible apps.
To help you detect ways to improve
the accessibility of your app during the development process,
Android Studio now contains built-in checks
to help make your layouts more accessible.
When using the layout editor, you
can now click on the Issue Report button
to see accessibility related issues detected
by the Accessibility Test Framework.
Accessibility test framework is the same library
that's used to detect issues with the Accessibility Scanner
app or when accessibility checks is enabled in Espresso tests.
Studio may even be able to offer suggestive fixes for key issues
of accessibility, such as ensuring
each UI element includes a description readable by screen
readers; interactive elements, such as buttons,
are at least 48dp large in both dimensions;
and sufficient color contrast to make text and images easier
to read and comprehend.
This integration currently only works for view based layouts.
And we're working on making it compatible with Compose
as well.
In addition to the new accessibility checks in Android
Studio, we've introduced a new API
that will make it easier for users to consume videos
with audio description.
Audio descriptions are narrations
added to video content that allow visually impaired users
to better understand what's happening on screen.
So if you work on an app that has video content
and the app supports descriptive audio tracks,
this API is for you.
Starting in Android T, there is now a global preference
that users can turn on to enable audio description
by default. With this default setting,
users no longer need to change the audio track
for every video consumed and can have a more unified experience
across all apps, especially when consuming content on TV.
This means that if your app serves video content
with descriptive audio tracks, you should now
utilize the isAudioDescriptionRequested API
to determine whether the user prefers
to play video content with the audio description track.
ANASTASIA SOBOLEVA: Hi.
I'm Anastasia, an engineer in the Jetpack Compose Team.
Let's go through some of the improvements that
were made in the Compose and Android Studio
to help you build an Android app that
is accessible to more people.
Compose is a modern declarative UI framework for Android.
It has been developed with accessibility in mind.
Where possible, Compose fills in accessibility
information by default. An example of this
is Compose setting the minimum touch target size.
Any onscreen element that someone can click, touch,
or interact with must be large enough
for reliable interaction.
On Android we recommend touch target sizes of at least 48
by 48dp.
In Compose, material components like checkbox, switch, icon
button, and other that can receive user input use
this minimum size by default. Let's
have a look at a radio button.
We pass in an onClick callback, which means that this radio
button can be interacted with.
Even though visually the icon is only 24 by 24,
patterns have been added so that the real size of the composable
has been increased to meet the 48 by 48 minimum touch target
requirement.
If however we set the onClick callback to null,
the size of the radio button is not increased.
In this case, the radio button is decorative.
It can't be interacted with and therefore can
keep its original size.
If the layout size of a non-material intractable
component is too small, Compose also has a fallback feature.
It will expand the size of the touch target only.
When the touch happens outside the boundaries
of the composable but within the minimum touch target area,
Compose will forward the touch input
to be handled by that composable.
Note that if there are multiple intractable components
adjusting to each other, there may not
be enough space to satisfy the minimum touch target size.
Therefore, we recommend you either
size the intractable component to be at least 48 by 48dp
or ensure enough space between the interactable components
so the expended such targets do not intersect.
To ensure consistency for all components,
you should be using the minimum touch target
property of the ViewConfiguration composition
local to define the minimum touch target
size for your components.
Compose uses the value of this property
to set the target size of the interactive components
and the minimum size of the interactive material
components.
I mentioned that Compose fills in accessibility information
by default. But that's not always possible.
Sometimes Compose as a framework doesn't have enough information
to fit into the Accessibility Services.
Accessibility Services are long-running privileged
services that provide different ways
for users with disabilities to interact with your app.
Google provides several Accessibility Services,
such as TalkBack and Switch Access.
To learn more about the commonly used Accessibility Services
on Android, check out android.com/accessibility.
Without the necessary information,
Accessibility Services can't meaningfully
convey to the users what's displayed on the screen.
In Compose we sometimes ask you to fill in such information.
Our APIs are written in a way that guide or even require
you to consider accessibility during app development.
For example, as a general principle,
you should label meaningful UI components like icons
that Accessibility Services can't automatically describe.
Therefore, in Compose we made content description
a required parameter in image and icon composables.
When the image is purely decorative,
you can choose to set the content description to null.
This API provides the most natural way for you
to express what makes the most sense for your use case.
We've seen how sometimes you need
to pass extra information to the Accessibility Services
and how Compose APIs help you with that.
Let's now take a look at how Compose uses this input
and how you can debug it.
In Compose, your UI is stored in a layout tree.
However, this layout tree doesn't contain any information
about the app's semantics.
Semantic information explains the meaning of your app.
You can think of it as an alternative representation
of the user interface understandable
by the Accessibility Services and the testing framework.
Therefore, in addition to a layout tree,
there exists a parallel semantics tree
that contains this information.
Compose uses this semantics tree to send information
to the Accessibility Services.
On Android it does so by mapping the semantics tree
to the accessibility node info tree
of accessibility framework.
A composable function that emits a layout node
will have its representation in the semantics tree
if semantic information is provided via the semantics
modifier.
Inside the modifier, you pass semantics properties.
Let's have a look in one of them, a role.
Roles are used in some of the components like buttons
and checkboxes to describe what they are.
The role variable is a convenient way
to set the role semantics property.
It uses the property delegation.
Looking at the set value message of the delegate,
we noted that all it does is set the value for the given key.
In our example, it set the Role.Button to the role key.
So semantics properties are key/value pairs
that describe the meaning of the component.
Some of the semantics properties in Compose
include focused, stateDescription, liveRegion,
and actions like onClick action.
One layout node might have several semantics
modifiers applied it.
Each of them could provide the same or different semantics
properties.
Semantics data from all semantics modifiers
is then collapsed into a single key value storage
that we call semantics configuration.
For example, when we call the text composable,
it already has the necessary semantics properties.
When we post additional semantics
with the heading property to mark this text the heading,
the resultant configuration contains the property
that was set implicitly inside the text
composable and the heading property
that we set explicitly.
This happens because the semantics applied
to a single node are collapsed.
Each layout node with a semantics information
will have a correspondent node in the semantics tree.
But such a tree might not necessarily
convey the correct meaning of the user interface,
because some of the SAP trees hold
the meaning of a piece of UI only as a whole.
Therefore, to ensure that such SAP trees are treated together
as a single unit, we need to merge
the nodes inside the SAP trees.
A typical example of merging in Compose would be a button.
Here it consists of two semantics nodes.
One node contains the semantics property
that describe the button as a click action.
Another node represents the text.
But only together these two nodes convey the meaning
of the Submit button.
Therefore, we need to merge these two semantics
nodes into one node.
Here is another example of the merging.
This UI represents a clickable card
with a few texts and an image.
If there wasn't a version mechanism,
an accessibility service-- a screen reader, for example--
would focus on each node separately.
But because this node as a whole conveys a meaning of a card,
we would like the screen reader to deal
with this node as a single unit.
To achieve this behavior, you should
pass through to the mergeDescendants parameter
of the semantics modifier.
Let's have a look how the merging is done.
The light gray rectangles represent the nodes
that don't merge descendants, while the dark gray
represent the nodes that do.
During the merging, the framework
will add semantics configuration of descendant nodes
that don't merge descendants into semantic configuration
of their parent node.
That way, the parent nodes with mergeDescendents set to true
accumulate the properties of the descendants
and their descendant nodes are then dropped.
Nodes that themselves have a mergeDescendents property set
to true will not be merged into parents.
As a result, the merge semantics history
will only contain representation of the layout notes
that merge descendants themselves.
In this example, the role and the inner bookmark button
use the clickable modifier.
Under the hood, this modifier sets the mergeDescendents
of the semantics to true.
On practice it means that the accessibility services--
a screen reader, for example--
will first focus on the whole container
and then separately on the bookmark button.
The merging is happening according
to the merging policy.
We define the merging policy separately
for each semantics property.
As you can see, the merging policy
resembles the reduce function of the Kotlin collection.
Here are a few examples of the merging policies
for the existing semantics properties.
For most of the semantics we pick the parent value
if one exists or otherwise use the child value.
For some though, we are the parent value only,
like in the role semantics property.
Some semantics properties concatenate parent and children
values into a list.
Some throw an exception if the property cannot be merged.
It means that the node that has the semantics property
must not set mergeDescendents to true.
As I said earlier, framework provides semantics information
whenever possible.
But you may encounter a use case where
you need to clear the existing semantics of the component
or its children and provide your own alternative properties.
Know that this shouldn't happen often
and might even be an indication that the user interface
you are building is not as accessible
as it could be in the first place.
So let's have a look on a typical example where
you might consider clearing the semantics.
Imagine there is a UI showing a long list of items.
A typical example would be in the email app or in use app.
In this example from the Jetnews sample app,
there is a list item showing details about an article.
Clicking on the list item will take you
to the Details screen for this article.
It also has a toggle button that allows
you to bookmark the article.
The list item merges semantics and so does the toggle button.
So if we use the screen reader like TalkBack,
it will first focus on the whole item,
and then it will navigate to the bookmark button.
If the list is long, it might not
be a good experience for the user.
Instead, we would like TalkBack to focus on an item
and have a bookmark option available as a custom action.
To clear the semantics of the bookmark button
to prevent it from being separately focused
by the screen reader, you should use a clearance
as semantics modifier.
Similar to the semantics modifier,
you may pass semantics property via this modifier.
This modifier prevents collapse in any semantics
of the given node coming after this modifier
and merging of any of the children of this node.
So we have a chain of modifiers as applied to a layout node,
including the semantics modifier.
And you pass a clearance as semantics in between them.
You can think of the result as if the resulting chain didn't
have any semantics coming after this modifier
in the modifiers chain.
Starting in Android Studio Bumblebee
we can inspire the semantics tree and semantics
configuration using the Layout Inspector.
Note that the Inspector provides two options for the semantics
representation--
declared semantics of the unmerged semantics
tree and merged semantics of the merged semantics tree.
To recap, in Compose, in addition to a layout tree that
represent the UI, there exists a parallel tree
that carries semantics meaning.
When you develop an app, make sure you test it
on several commonly used accessibility services.
If you encounter any missing semantics information,
you can provide it either via the common accessibility
APIs available in Compose or by applying techniques
discussed in this talk.
To learn more about the most common accessibility
APIs available in Compose, go check the Google I/O
2021 session, Designing for Accessibility in Android Studio
and Jetpack Compose.
And that's it from us for now.
Thanks for watching, and be sure to check out
the resources included in the links below
to learn more about what we've covered in this video.
[MUSIC PLAYING]