Hello everyone, my name is Lotus Zhang.
I’m a PhD student at the University of Washington.
Today, I’m presenting our paper: Exploring Interactive Sound Design for Auditory Websites,
on behalf of my coauthors Jingyao Shao, Augustina Liu,
Lucy Jiang, Abigale Stangl, Adam Fourney,
Meredith Ringel Morris, and Leah Findlater
As more people began to adopt voice technologies,
accessing online information through audio has become increasingly common.
More voice interfaces now support users to listen to full webpages
However, the current experience of LISTENING to online information is rather limited
compared to the experience of visually browsing the internet.
Online content are usually carefully arranged and designed into websites,
such as through the manipulation of visual layout, font, color and pictures.
In contrast, audio delivery of online information
often uses a single synthesized voice to speak almost all content,
collapsing rich visual design into a static audio style and a set of serialized texts.
This gap is surprising,
as sound is innately communicative and expressive.
Besides speech, voice utterance, music, and nature sound, all convey important information,
including emotion, atmosphere, and physical properties of the the sound source.
Sound design practice also has a long history,
with extensive practical knowledge,
knowledge that is often overlooked in HCI research.
This paper, therefore, explores aspects of professional sound design practices
that can be incorporated into auditory presentations of websites,
which we refer to as auditory website.
We conducted a set of interviews and design activities with professional sound designers,
with a focus on the following research questions:
How do sound designers conceptualize auditory websites?
What suggestions do they have for auditory website design?
What factors influence their design choices?
We recruited 14 professionals through two freelancing platforms:
Upwork and Fiverr.
Participants’ experience with sound design range from 4 to 20 years,
with an average of 12 years.
All participants self-reported to have some experience with web design,
which ensures a basic understanding of website user experience.
We conducted a three-phase remote study with each of these professionals
First, we had a 45 minute initial interview to introduce the idea of auditory website,
and prompted participants to envision and comment on potential sound design techniques for it
We then asked participants to create an audio clip mockup for a specific webpage during a two and half hour design activity.
including the homepage of the New York Times,
the Titanic page on IMDB
and a calculator page on Walmart.com.
Last, in a 45min final interview, we asked participants to describe their design and reflect on their approach.
We then analyzed participants’ audio mockups, design plan, and interview responses
to understand how they conceptualize sound design on auditory websites.
To give an example of what the data is like,
here is a 15 second preview of P9’s mockup of the New York Times website:
[Piano music]
[Female-sounding synthesized voice] The New York Times
[Male-sounding synthesized voice, orchestra music] The New York Times is made possible by the following:
[Female-sounding synthesized voice] Tony award-winning broadway musical, Oklahoma!
[Male-sounding synthesized voice, light music] Navigation bar...
Our analysis identified aspects of auditory websites that participants intended to improve through sound design.
We synthesized these aspects into five design considerations:
aesthetics and emotion, user engagement, audio clarity, information dynamics, and interactivity.
First, to deliver the aesthetics and emotion of specific website content,
our designers suggested to use ambient sounds and voice variations to create a specific atmosphere
For example, door slamming or witch cackle could be used for presenting spookiness
[door slamming and witch cackle sound]
hustle and bustle from the street gives the vibe of big cities
[street sound]
Many also directly used music and voice tones to create atmosphere,
such as using the song “My heart will go on” to set the theme for movie Titanic [music "My heart will go on" playing]
and an “older gentleman” voice for serious storytelling
Second, to engage users with prolonged audio interaction,
our designers considered avoiding unpleasant audio and introducing variations.
For example, the variation can be in the synthesized voice, such as its personality or gender,
and can also be in the background music.
Participants also emphasized on the importance of audio clarity and considered improving it
through soundtrack arrangement and synthesized voice quality control.
For example, they suggested using speech rate and accent that fit user’s listening preference
as well as eliminating distracting background soundtracks.
The forth consideration is information dynamics.
In visual media, important information may be presented in large, eye catching fonts.
On par with that, our designers suggested to use distinctive audio composition,
such as bold sound elements and voices that stand out, to indicate that certain information is important.
Last, all designers wanted to support the interactivity of auditory websites,
such as its navigability and efficiency.
They explored various design techniques for this consideration.
For example, using short sounds that people can relate with
to more intuitively represent different functions or sections of a website,
or to add a unique soundtrack for each section,
so that users can hear how deep they are in the website. Please refer to our paper for more detailed ideas.
Please refer to our paper for more detailed ideas.
Besides the five design considerations,
I'd also like briefly present challenges experienced by our designers
and related next steps that we see as potential valuable future directions
First, most websites were created with the assumption that they will be delivered visually,
and therefore its structure and content can be visually-oriented.
For example, maps and site overviews are inherently visual content,
and are difficult to convert into auditory presentations.
Future work should think about design processes that are more modality-neutral. For example:
coming up with content that can be easily represented through both visual and auditory format
or developing design workflows and design tools that allow SOUND designers to work alongside visual designers
Finally, while our study explores auditory website design ideas from designers’ perspectives,
a critical next step is to understand how end users feel about the ideas we have identified.
In summary, sound designers in our study prioritized creating
aesthetic, expressive, and engaging listening experience for auditory websites.
It is in contrast to past voice interaction research focus on functionality and usability,
but echoes recent argument from the field of sonification that
aesthetics has immense value in supporting listeners’ meaning making of audio content.
Our participants’ creative explorations of sound design ideas provide new insights for future research to reference.
Their challenges also point to exciting research directions around auditory website design support
that we believe will lead to more enjoyable experience for people who access websites through audio.
Thank you for listening to our paper presentation!