Hello, my name is Laxmi.
I am a PhD student in Human-Computer Interaction lab at UC Merced.
I am going to present the work exploring the acceptability of speech and silent speech as active input
methods in private and public places.
Silent speech input interprets users' lip and tongue motions into text.
It is a promising alternative to speech input in contexts where the acoustic signal is unavailable
or unreliable.
Though the method shows promises, very little is known about peoples’ perceptions regarding
using it to interact with mobile devices in private and public settings.
In our exploration, we study social acceptance of speech and silent speech input considering
factors such as users’ and viewers’ perspectives towards using the methods in different locations
and among different audiences.
We first conducted an online survey on Amazon Mechanical Turk with 71 participants to explore
the social acceptance of both input methods.
Participants were asked to watch a set of video clips where users were using either
speech or silent speech to interact with mobile devices.
We then asked them to provide their feedback on social acceptance of these methods considering
different locations and audiences.
The social acceptability for both methods from users’ and viewers’ perspectives
were significantly different across locations on mobile devices as users preferred using
silent speech as it is less noticeable.
But we did not find any significant difference between two methods across audiences.
In subjective feedback, participants expressed their interest in using silent speech input
as it is more subtle and provide a high degree of privacy and security than the other method.
The findings indicated that there are accuracy and interaction factors that could influence
their attitude towards using the method.
For instance, silent speech recognition accuracy could be one of the key factors in adopting
the method as it could be highly error prone.
We conducted a Wizard-of-Oz study with 12 participants to explore users' attitudes towards
recognition error in speech and silent speech where we injected 0%, 5%, 10%, 15%, and 20%
misrecognition errors.
During the study, participants entered short English phrases using speech and silent speech
at varying injected error rates.
We then collected user response to the error rate for two input methods in a questionnaire
that asked them to rate their willingness to use the methods.
Results revealed that 0% and 5% error rates yielded the highest and 20% error rate yielded
the lowest willingness to use ratings for both methods.
Interestingly, for 10% and 15% error rates, the willingness to use ratings for speech
dropped at a higher rate than silent speech.
This suggests that users were willing to tolerate more errors in silent speech to uphold privacy
and security.
Silent speech input recognition on mobile devices depends primarily on capturing users'
tongue and lip movements via the front camera.
Thus, providing appropriate real-time feedback on input recognition is critical for the acceptance
of the method.
We conducted a user study to compare two different types of feedback mechanism for silent speech
input.
The first is "abstract" that uses a red blinking icon and the second is "detailed" that displays
real-time video of users.
We conducted the study with 12 participants.
During the study, participants entered short English phrases using silent speech with both
feedback methods and then responded to a questionnaire that asked them to rate various aspect of
both methods.
Results show that users find both feedback methods effective but the video feedback significantly
more informative than abstract feedback.
Yet, they were slightly leaning towards using the abstract feedback since they found it
to be the least intrusive and significantly more private and secure than the video feedback.
Our findings confirm that silent speech input is more socially acceptable as it is subtle,
more secure, and less attention-seeking than speech input.
Furthermore, users are willing to accept more recognition errors with silent speech input
than speech input.
Our final study showed that users find a commonly used video feedback and an abstract feedback
effective, but the latter to be significantly more private and secure than the video feedback.
Thank you
I am happy to take any questions you may have about the work.
Also, feel free to ask us questions or comment on our work via our social media channels.