yeah it's true and like it's all about devices can you imagine becky and i trying to sleep at
night and we've got all of these chords charging every single thing yeah in our digital lives
we're no strangers to smart devices that track out every move search and like but
then i logged onto amazon to buy dog food and got an ad for their new fitness tracker halo
it counts your steps tracks your sleep and analyzes the tone of your voice so you can be
more positive in your day-to-day interactions tone of voice analysis is a step too far that i find
deeply disturbing so naturally i bought a halo to test it out i convinced my maker friend becky
to get one too so we could team up to ask a few questions one is does the technology work
the second is is it useful and then the third is is this something that we need to measure
actually before we get to that though we have some unboxing to do i got the gray color which
is probably called smoke or mist or something like that oh it's called winter there you go
nothing can be gray anymore the amazon halo is a pretty typical fitness tracker
it has an accelerometer heart rate monitor temperature sensor and they all provide you
feedback and scores on your sleep patterns and physical activity it's bigger than i would have
expected it has some nice possibly compostable packaging here which you know just offsets all
of amazon's other activity in every corner of the world um taking it off here we can see that
it has a velcro band innovation coming from the space age that's nice to see
it has a screenless wristband that communicates with an led
light and it also has a microphone and no digital device is complete without the most
important most essential most smart component get the app oh i have to download their app
i'm kind of resistant to giving every single piece of information to amazon but for everyone out
there wondering my birthday is the 3rd of march please send me gifts thank you how tall am i just
wants to know everything about me the app somewhat like the tracker itself is both intrusive and
intriguing okay so on the app it says you can sound hopeful elated hesitant happy worried or
confused angry or affectionate how can it even tell what all of these different emotions are
through the tone of your voice when that varies so much from cultures and accents and intonations and
cadences and everything i'm just i'm so stressed that that this device is claiming to do this
because it can have a big impact on how people change their behavior or live their everyday
lives and like where is all of this science coming from this unboxing just got very serious
i had to dig deeper so i decided to go old school and get an analogue sounding board
you know a conversation with a friend how am i being skeptical and stubborn and now i'm amazed
and you're also embarrassed this is becky stern a maker and educator who has way
more experience with this kind of tech than i do so much so she actually tore the band apart
the training data had to involve people characterizing other people's speech right it
had to have come from not a and then the machine takes the results from the human saying this
speech was positive neutral or whatever and and so it it's interesting psycho psychology to think of
what the training situation was like because those people who are sitting there analyzing the tone of
the sample people that they're listening to or the sound recordings that they're listening to
they have a bias too i want to meet them and this it leads to a lot of questions about how
the band and tone analysis actually works how does the tone feature work
ah they try to cover it up how it works right they try to make you just trust it say we know
we know what your tone is but it's been trained it's a artificial intelligence machine learning
situation where they've trained uh their app on some kind of set of data based on
what they must have collected it so people talking and then people saying that they
thought that their tone was this this or this the halo band analyzes your voice using two factors
positivity and energy according to an amazon spokesperson a group of employees listen to tens
of thousands of vocal samples and decided whether the voice was happy or sad and if it was tired or
excited and ai then used this data to learn how to classify tone of voice but when you speak
the ab says right now i'm passionate excited confident delighted well this is you now
it confuses me for vanessa because it's not very good at this because we said we sound so similar
the two of us yeah so i don't know i i get in general how they trained it yeah but we don't know
who trained it we don't know how much data and we don't know how they got these specific words where
they are on the chart yeah not only are these two parameters of positivity and energy both vague and
inadequate this tech has another flaw that i take personally there was one article where an amazon
representative said that if you have an accent it will be less accurate but i find that hilarious
because what what accent like literally everyone has an accent yeah you may recall some videos i've
made over the years on people's bias towards tone and accents how people rate low pitched
male voices as more attractive low pitched female voices as more believable or people with foreign
accents as less credible so if there's cultural buyers at play in how the tone analysis works
is that still useful in order for halo's tone analysis to be accurate and useful it needs to
account for what different tones mean in different parts of the world there's not necessarily a
default tone for positivity or negativity or high energy or low energy so it would need to take
data points from people genuinely expressing those tones in all different places across
accents and cultures and cadences and where do we even get started on sarcasm i'm trying
to think of what cultures are like really known for faking their interest or concern or america
america especially southern americans right or like bless your heart means like screw you really
often often it does the halo appears to be modeled around a west coast american accent and cadence
so if i can't even grasp the emotional charge of speakers on the other side of the same country
what hope do people from all around the world have in an accurate tone analysis
i think part of it is australian slash british politeness you know like i was pretty
upset about something earlier today like someone lost one of my packages and i was
like i'm so sorry to bother you i know you're really busy but i think you've misplaced this
and this was like you you're being friendly and amused i'm like i'm pretty angry that you lost
my package actually but i'm not gonna yell at you because but like what i wouldn't do that
the band thought i was positive all the time in all kinds of scenarios so i started to wonder
what other accent might be rated as negative even when they're intending to be positive all
right should we train this for arnie yeah let's do it okay learning does not make one learn it
there are those who have knowledge and those who have understanding
yeah we trained the amazon halo to recognize arnold schwarzenegger's voice you tell me
who is your daddy and what does he do school i'm going to be your sheriff
you're going to be my deputy trainees overbearing aggressive irritated come on stop whining
when you've fallen down a rabbit hole you can always go a little bit further you like
discipline angry so the amazon halo jury is in vanessa hill's tone is overwhelmingly positive
arnold schwarzenegger on the other hand is just a plain old negative nancy but arnie aside is tone
analysis actually necessary is that something that we need to measure i find it it's useless
for me for policing my own tone as if pull you know as if policing your tone is a thing
you should be doing but i feel like that that's what they want you to use this device for like
what am i like and how could i be different it wasn't a very positive psychological experience
for me to really to see this tone data even though it's saying that i'm amused happy uh it
it didn't give me any like useful information and this is perhaps an unintended consequence
of technology like this we're already a society becoming hyper aware of our daily movements from
physical activity to sleep what's the impact of us having more information on how we speak
which isn't as objective well i looked at the comments on the reviews on amazon just to try
to see what people were saying about toner voice and it wasn't great some people were saying that
i sounded condescending all the time that was from a woman as well there was a man who said it always
says i sound sad and that comment or review just resonated with me so much because psychologically
that can have a pretty huge impact on you like if you're using something that is billed as like an
objective kind of expert device and it's telling you you're sad that can affect your behavior in
certain ways like that it can affect your mental health in certain ways like all of this feedback
is training us to be more positive and more high energy and why is that necessary like what if
i just want to be low energy negative person or even a lot of comedians who are genuinely
really funny have a kind of low energy delivery on things yeah it's like a deadpan yeah like some
people just kind of speak like that experts say that worrying about our tone of voice during a
social interaction instead of focusing on the interaction itself may be detrimental
making it harder to meaningfully connect because it could also lead you to avoid
certain conversations like if you were having a conversation with your mum or dad or significant
other or husband or wife or whatever and it was saying that you were being like
sad and condescending but it was about a really important topic it might make you less likely to
have those important conversations because you're just like worried about how you would come across
so this excessive self-monitoring is bad for us in a lot of ways like if it tells us we're sad
we think we're being sad and we feel sad and it can manipulate our behavior but then it can also
make us avoidant as well which definitely isn't a good thing the one thing that i thought this
may be could be useful for is someone who has autism or is neurodivergent in another way
and they don't think they're picking up on a lot of cues and social situations but they
want to practice or try or like see how they sound if they can't really gauge that
maybe they could use this but i really don't think that that is a good or necessary use case
even practicing for a ted talk maybe yeah i would like you to answer my questions does this work no
is it useful even if it worked it wouldn't be useful should we be measuring something like tone
not with our bias training data so we have three nodes i need a bell that you can just put it in
post again three nodes in theory tone analysis could be useful but in practice it's not useful
when it's not accurate and we have almost no idea how it was developed something that would help me
believe that this is accurate is for them to sh to share the training data set with us as far
as i'm concerned tone of voice analysis enters the territory of excessive self-monitoring with little
actual benefit to your behavior or our brains there are some things that we can track that
are actually useful like your steps or sleep and it's okay for all of us to say let's stop there as
the great arnold schwarzenegger once said in total recall what's the point of being on this earth if
you're going to be like everyone else or something to that effect i don't have total recall so sorry
if you'd like to see the insides of the halo becky stern did a teardown over on her youtube channel
please head over there with me and subscribe right now or otherwise you'll have to sit
through her schwarzenegger voice impressions you have been warned yeah i think then what i
am going to do is uh give it away to someone in the comments whoever has the most liked comments
oh on this video tell a funny joke um i'm going to message them if you can still dm people on youtube
and ask if they want this so this is just a competition uh that's now running on my
youtube channel lowkey competition most liked comment is gonna win this oh i hope i get it
straight from my wrist to yours it was the best of times it was the worst of times it was the age
of wisdom it was the age of foolishness it was the spring of hope it was the winter of despair