0:00
Hello, I'm Johnna Blair from Penn State University,
0:02
presenting our work --
0:04
Understanding the challenges of using smart assistants
0:07
for Deaf and Hard of Hearing users.
0:10
Smart assistants use the user's voice to perform tasks
0:13
and answer questions in natural language.
0:15
These include smartphone assistants like Siri,
0:19
and smart speakers like Amazon Echo.
0:21
Their pervasiveness and recent affordability
0:24
have made them a key consumer device.
0:26
They also have a unique adoption pattern in that
0:30
older adults have been the largest group
0:32
of first adopters for smart speakers.
0:36
Smart assistants provide hands-free use,
0:38
simple naturalistic voice interactions,
0:41
and the ability to connect to other devices
0:43
for home automation.
0:45
They also don't require fine motor movements
0:48
or reading small text
0:50
and can be used at a distance or from a fixed location.
0:53
Because of this, they have increased
0:55
accessibility and independence
0:57
for those with low vision or mobility concerns.
1:00
However, the reliance on voice interactions
1:03
inherently poses a challenge
1:05
for those with hearing loss.
1:08
20% of individuals in the US across all ages
1:12
report some degree of hearing loss.
1:14
But older adults, those early adopters of smart assistants,
1:18
are also disproportionately affected by hearing loss.
1:22
And this shows a disconnect between
1:23
the current design and the needs of its users.
1:27
So to explore this disconnect, we asked--
1:31
how are those with hearing loss
1:33
currently using smart assistants?
1:35
and what challenges do they experience?
1:38
We first conducted in depth interviews
1:40
with profoundly deaf participants
1:42
and then surveyed those with mild
1:44
to severe hearing loss to see how pervasive
1:47
some of these insights were across the
1:49
wider deaf and hard of hearing community.
1:52
Overall, participants showed
1:54
high rates of daily use,
1:56
and their most common tasks
1:58
mirrored that of general users,
2:01
apart from music use.
2:04
The leading challenges involved
2:06
the default voice qualities
2:08
of smart assistants.
2:10
Alexa and Siri both use female personas
2:14
that come with a higher pitched voice.
2:16
However, this frequency range is
2:18
often problematic in age related hearing loss
2:21
and is also difficult to capture with hearing aids.
2:25
As such, many wanted their device
2:27
to provide a lower pitched voice.
2:31
Users were also either unaware
2:33
of slow down commands
2:35
to change the default speed or
2:37
found this disrupting to do mid-task.
2:41
Additionally, using a louder voice in smart assistants
2:44
did not necessarily address these other issues.
2:49
Device feedback was also a challenge.
2:51
Error messages were unclear,
2:53
leaving users unsure what went wrong
2:56
or how they were to go about fixing it.
2:58
Most relied on visual feedback
3:00
to go along with the voice responses
3:02
and preferred devices with screens that
3:04
could support this, such as smartphones.
3:07
The light messages provided by smart speakers,
3:10
while promising in concept,
3:12
were seen as vague or confusing
3:14
in their current state.
3:17
Users reported that smart assistants
3:19
were less accurate in interpreting
3:21
their commands when they were given
3:23
without their hearing aids in use.
3:26
Experiences were also not as natural
3:28
or hands free as expected.
3:30
Many of their activities were interrupted
3:32
by needing repeat response commands,
3:32
or by moving closer to the device
3:38
to hear it more clearly.
3:40
Users also felt unsafe using smart assistants
3:43
while driving because of the additional attention needed.
3:47
Also, using smartphone assistants
3:49
in public often came with too much competing
3:51
background noise to interact the way they wished.
3:55
What can we as the UbiComp community do
3:58
to make smart assistants more hearing accessible?
4:02
To start, we can provide easy to use,
4:05
full voice customization options for pitch, speed, and volume.
4:10
We should also develop systems that can
4:12
dynamically tailor smart assistant voices
4:15
to the user's current range of hearing.
4:17
Such a system can leverage audiograms
4:20
to determine this ideal hearing range.
4:23
This can help smart assistants adapt
4:25
to the user over time as their hearing needs
4:28
continue to change.
4:29
There's also a need for research to
4:32
reduce background noise for different environments.
4:35
Supporting Bluetooth connection similar
4:37
to traditional devices, as well as keyword emphasis
4:41
could be particularly useful in this case.
4:44
By training new systems on a wider range
4:48
of voice types, future smart assistants can
4:51
accommodate more hearing diversity
4:53
and different speech patterns.
4:55
Moving forward, we can explore
4:58
gesture recognition AI as a new interaction modality
5:02
to make smart assistants more accessible to those
5:06
who primarily use signing languages as well.
5:10
With greater smart assistant accessibility,
5:13
these devices can assist users in new ways.
5:16
They can help address current life challenges
5:19
such as facilitating large work meetings
5:21
and supporting communication in public settings.
5:24
More broadly, improved light feedback in smart speakers
5:28
can improve device communication for all users.
5:32
The insights from this work also show promise
5:34
for using smart assistants as a means to support
5:37
speech therapy applications, in the future.
5:41
So overall, while smart assistants improve
5:44
accessibility for different use cases.
5:46
There's a considerable gap when it comes to
5:49
the needs of individuals with hearing loss.
5:51
Our work is a step towards addressing this gap.
5:54
We believe the ubicomp community can
5:56
play a strong role in making smart assistants
5:59
truly inclusive in the future. Thank you.