Hello, my name is Kristen Scott, with the Madeira Interactive Technologies Institute.
I will be presenting the paper "Human, All Too Human": NOAA Weather Radio and the Emotional Impact of Synthetic Voices
To give you a brief outline of the presentation, I will start with the context of our interest
in synthetic speech on the radio Followed by a brief discussion of an exploratory
focus group we conducted, and finally I will explain the details of
our analysis of relevant YouTube videos and comments.
As part of an EU-funded community radio project we are exploring integrating
text-to-speech (or TTS) capabilities into an open source technology stack for
low-power FM radio stations. Our goals with this proposition include
increasing accessibility to information for people without internet,
turning useful data into content and providing support for chronically understaffed
community radio stations.
Studies show that reactions to synthetic voices are subjective and influenced by a
variety of factors, like the content, perceived speaker traits and pronunciation
In a traditional listening test, Baird et. al
found a correlation between perceived humanness and likeability of synthetic voices
In practice, however, overly human-like features
in conversational user interfaces, such as virtual companions, or chat bots,
have sometimes been found to lead to negative reactions, such as
unrealistic expectations and frustration, Views that the interface is pre-programmed
and inauthentic an people being put off by a device presenting itself as a human,
considering this a "lie", "dishonest", or "fake"
We conducted an eight person exploratory focus group about the concept of synthetic
speech on the radio. Participants were introduced to the concept
of generating community radio content using synthetic speech and content pulled
from the internet They were then given the opportunity to experiment
with a variety of voices and content. Three themes emerged from the exercise
and discussion. Concerns about the level of connection possible
with synthetic voices was a prominent theme. There was general agreement
that radio was meant to be a kind of parasocial experience and that synthetic voices
may be antithetical to this role. Participants tended to focus on a need for
increasing the sense of humanness of the voices. Some suggested increasing tone variability
and emotion. Other suggestions focused on introducing imperfections such
as "uhms" and "ahs", or by adding ambient sounds such as pages flipping or a pen dropping.
Participants also reported specific conceptions about the types of voices that "should"
be used for reading certain content, for example, preferring "serious" voices for news,
and "friendly" voices for stories.
Now I will move on to our analysis of the use of synthetic speech on NOAA Weather
Radio.
NOAA Weather Radio of the National Weather Service of the United States of America
We chose to focus on this topic as it is an example of a highly visible and regularly
used application of synthetic voices in a radio
context. The content of each local station is delivered
almost entirely through synthetic voices. As of January 2014, the NWR network included
over 1000 stations that covered about 97% of the U.S. population. Station frequencies
are outside the normal AM or FM broadcast range and require a special receiver
or weather radio. These radios are widely available and in some areas built directly into homes.
In the late 90s a computerized voice nicknamed "Paul" became the voice of NOAA
Weather Radio, In 2001, other voices were added, including
a female voice, “Donna”, with the male voice “Tom” becoming the primary voice.
In 2016 the National Weather Service rolled out a new system, which utilized a new
voice, and did not include these previous voices. This was a modern synthetic voice
created by the company Neospeech. It was also named “Paul”. This voice was touted as
a much improved version of the original Paul
and came to be referred to by NOAA and its listeners using various nicknames,
such as "Perfect Paul", "Paul Jr." and "New Paul".
We can listen to these voices now
[audio clips being played]
The introduction of that new Paul voice was actually met with some negative reactions from listeners.
This voice change is a topic of many of the videos and comments that are included in this study.
Interestingly, YouTube has become a gathering place where communities of NOAA
Weather Radio enthusiasts post audio and visual clips of the radio broadcasts.
Discussion in the comments focuses on various aspects of the technology and the
content, including on the synthetic voices themselves.
In this study, we collected relevant YouTube videos and their associated comments in
order to analyze reactions to the various synthetic voices. We accessed videos using
the YouTube API by conducting a search using keywords
"NOAA", "weather", and "radio" along with the voice names.
Twenty nine videos, with 513 total comments were included in the final analysis. The
three authors coded and analyzed the comments using an affinity diagramming process.
Coded comments were iteratively arranged into a series of sub-themes and larger
themes until agreement was reached on the characterization of the commenters'
reactions to the NOAA Weather Radio synthetic voices.
Statements referring to the various synthetic voices as real individuals - with
personalities, private lives and relationships occurred frequently in the comments.
These were generally used to express the emotions people feel around the voices,
and conveyed the way they conceptualize the voices in their own minds. The change
from the Tom voice is commonly referred to as if it represented the death of a known
person or friend. There was repeated use of the term "RIP", along with many
mentions of "missing" Tom. There are occasional instances where the voices
refer to themselves within the broadcasts in a fun or joking way. There are
no comments suggesting annoyance that these overtly "human" statements are fake,
manipulative or in any way off-putting, instead there was acknowledgement of, and
appreciation for, the humans behind these moments. We can listen to an example.
[audio clip]
Commenters also share stories of their memories of the voices, which come across
as deeply ingrained and nostalgic. The voices trigger associated childhood memories
as well as distinct memories about the voices themselves.
This sense of nostalgia may explain the highly positive feelings declared towards the
Original Paul voice, which is objectively of lower quality than any of the later voices.
Videos featuring this voice receive overwhelmingly positive and appreciative comments.
The connection of the voices to severe weather event warnings means that they are
potentially associated with negative, disturbing or even traumatic memories. The
voices themselves, including the generally loved Tom and other older voices, are
sometimes referred to as creepy and frightening. However, this eeriness is not
necessarily viewed as a negative feature of the voices. Commenters specifically
stated that a "cold", serious, even "creepy" voice is preferred. Friendliness and
increased humanness are not particularly valued highly in these voices. Commenters
describe the largely disliked New Paul voice as sounding "like some cartoon
character", not being serious enough, sounding "so human", "dumb", and "too easy to
tune out".
Comments often mention pronunciation and prosody (meaning speed and intonation).
Comments that reference these features after Paul was
introduced, are uniformly negative, with the majority of complaints being
about the long pauses. Slow delivery with long pauses is poorly received and in some
cases described as dangerous where extreme weather is concerned.
Mispronunciations are identified and seized upon by multiple commenters when they
occur in the videos, with some even mentioning other mispronunciations they have
witnessed aside from those featured in the video under discussion. Mispronunciation
is a common complaint about New Paul and is often treated as highly unacceptable.
However, mispronunciation is not always viewed negatively. In some cases a
mispronunciation is forgiven or even appreciated as a quirk of a given voice.
Let’s move on to some takeaways in terms of the design and use of synthetic voices
for the radio.
While participants in the exploratory focus group expressed misgivings about the
viability of TTS technology being used on community radio, citing an inability to connect
with a synthetic voice, the comment data we examined shows anthropomorphism of,
nostalgia for, and an emotional connection to synthetic voices.
This occurs even though the synthetic voices
that have been used by NOAA Weather Radio are of varying levels
of quality and humanness, and are not presented as pseudo-human entities, or made to
seem particularly “real”. Still, it is clear that introduction of synthetic
voices onto the radio is a sensitive matter and care must be taken to avoid off-putting
missteps such as mispronunciations of critical named entities (like names of places),
slow utterance rates, and inappropriate voice-to-content matches. But it also is interesting
to note that not all synthetic speech errors are viewed negatively.
A familiarity and long term exposure, along with the intimacy of the medium of radio
itself, appear to be strong contributors to yielding positive reactions and a sense of
attachment to the synthetic voices. This effect, what might be termed "experiential
exposure" to synthetic voices, as observed in the NOAA Weather Radio comments, is
not something that was anticipated by members of our initial focus group.
We conclude that future research in the area of text to speech technology should not
single-mindedly focus on generating more human-like voices or voice experiences and it
is possible such efforts can actually result in negative reactions and become a hindrance
to ongoing engagement with the synthetic voice. We suggest that future work in speech
synthesis should focus on improved pronunciation, speed, and suitability of voice to
content, while not assuming that increased "humanness" is a necessary precursor to
these factors. In terms of text to speech applications, we suggest acknowledgement of
the non-human qualities of the voices and their "behaviours" as an alternative to
attempting to present them as a pseudo-human.
This is a list of the papers I have referenced here today.
And I thank you very much for your time. My name again is Kristen Scott and thank you
to my co-authors Simone Ashby and Julian Hanna