thank you very much everyone good
morning Budapest how's everyone doing
doing
hmm so before we get started I'll give
you a little bit background I guess I
will not so I will not do so good as
Paul did but um my name is Mauricio
Palma I was born and raised in the
beautiful country of Costa Rica and I've
been living for the past 13 years in
Hamburg Germany
not because of the lovely weather Eddie
it is an amazing city don't get me wrong
I work as a product engineer at
sahasrara and you will see me also
wearing multiple hats as the co-founder
of would like ocean where we craft swim
or out of ocean recovery fishing nets I
have to warn you though this
presentation is going to be filled with
personal anecdotes so be ready and the
first story is about how I gave my very
first conference talk it was the
diseases called Budapest two years ago
and I just have found out that David
yesterday said on the CSS Cohn he did
his first top two here four years ago so
today it is a very special humbling
moment for me and I'm so happy to share
it with you if you have ever held a talk
before the first time it is a very
exciting nerve nerve wracking experience
and as you can imagine I was super super
nervous I thought I was gonna die
especially because all these amazing
people then some of the speakers you
have followed during your career and I
can't forget how Dennis mission of came
to me and gave me some really comforting
words and I think I had just this
terrifying look in my face probably but
I did my best then and I think the
presentation came all right but my point
here is this is a very special
conference and I am very very
excited to be here and share this
presentation with you if you have your
own ideas if you want your voice to be
heard this is the best platform to do it
so submit your ideas and you will never
forget regret it
and for that reason I want to thank the
organizers for having me I want to thank
you for being part of this amazing
community and I hope that we can use
next two days to get inspired to meet
each other to learn new things and for
that I hope let's give it up for that
[Applause]
the web can be an amazing place a place
where we can collaborate where we can
learn where we can be creative and it
has an inherent open nature to it
information is accessible to everyone
and for me was especially important
because I'm a self-taught front-end
developer now by race of hands how many
of you are self-taught developers that's
a lot of us and now imagine this
information would be only accessible to
a certain amount of people that you were
not invited to be part of it and this is
actually what we're doing every single
day when we're not thinking about
accessibility and the applications will
build in today we want to invite and
empower people through emerging web
technologies and I love being a front
developer I love my craft and working
with the latest technologies learning
about new paradigms and bringing
projects to life
and I think we share the same passion
but what I have learned in later years
is that my work should serve a higher
purpose that just my own gratification
and I know web accessibility is not an
easy endeavor especially if you're
working on large-scale projects with a
lot of decision makers can become a real
challenge you get to hear stuff like
accessibility it's not important for us
it's not relevant for our target market
and these kind of comments are
frustrating and yet it is still our
responsibility to remove these barriers
to teach people and try to promote and
implement accessibility we as developers
have a collective responsibility for the
Caleta for the applications we were
building today even if we are aware of
it or not think about it for a second
most of us have seen hundreds thousands
of websites have gone through a ton of
user flows and we have this toolbox full
with buzzwords and my point here is it
doesn't matter how badly design a user
interface is we still make our way out
but this is not how the rest of the
world is interacting with the
applications widow not at all
we are working in a bubble trapped in
our own bias and like I mentioned a lot
of personal anecdotes but don't worry we
have some technical stuff too
there were two events in my life that
made me realize about the importance of
accessibility so I had to learn the hard
way and the first one was a painful one
a family health issue my uncle in law he
was found on the floor dehydrated and
unconscious and he was probably on the
floor for a couple of days so today
we're very happy that he's back home but
you have to consider that he's an
elderly person
leaving by himself so before the
accident he completely relied on the web
to get pretty much everything done he
will buy his groceries online pastries
would be order for cut family coffee
dates he will take care of his bills and
finances and unfortunately after the
person after his hospital release he
know he was diagnosed with with poor
vision and memory loss
and after he release he called me and he
said Modi see I think my computer is
broken you know that when computers are
broken family are calling normal stuff
and as I'm trying to order some
groceries and I think my computer is
broken so I went over there to see what
was the problem
and so his computer was working just
fine and I asked him can you please show
me the problem and this is the moment
that I realized and I will never forget
he couldn't recognize and Add to Cart
button he couldn't recognize or
differentiate between copy text or input
fields he could neither gave to release
ID at all and I was very sad because I
realized that I swapped developer was
not being responsible enough with the
applications I was building until that
moment
but we can actually make better use of
the current accessibility tools to help
people like my uncle in law get things
done online we can even go further
further and think about all these web AP
is that we can use to our advantage to
actually make the web a more open place
but what I have observed among us
developers sometimes is that
accessibility is perceived as just a
checklist a list of things given by the
web content accessibility guidelines for
us to implement and it reminds me a lot
to the same attitude we have for
front-end developer for alders front-end
development for all their browsers you
know when you have been working on a
project and the time has come for you to
check it in an older internet explorer
think about it for a second and and my
conclusion to that is that we as humans
we tend to avoid things that we don't
understand and that's our nature and I
think that's why accessibility is being
perceived that way but what if we start
considering accessibility as the core
concern of our application now let me
give you an example in Europe alone
there are 30 4.4 million people I mean
let me correct that adults with
disabling hearing impairments a
disabling hearing impairment it is one
that is greater than 35 decibels and
every single day we denied deaf and
hard-of-hearing people access to our
personal and professional lives and
during my research I found this app
called Ava that I would like to show you
and I think the team challenge this
notion with their work
and let me show you what they're doing I
am
one day is too many people down this can
get really tough
people talk so fast it can be pretty
overwhelming
there's too much going on
ava is awesome to see what people say
with one tap I invite people and they
join me with their phones
I knew Ava to attend meetings at what
follow what the teacher says in class
[Music]
what kind of sound people Wendover got
to look at me
baby I will transfer about anything
oh well almost everything
and that for me was very inspiring I'm
not related to them I don't even know
the team but I think that's a really
good example of when you recognize
exclusion and you be and you become
empathetic towards people with death and
hearing impairments then they were able
to come up with a meaningful solution
maybe it's not the best one but it is a
good start and that's why we're here
talking about it today because we can do
this kind of things on the web natively
and I mentioned two events this is the
last one I promise and the second one I
had a sport injury so I dislocated my
arm I couldn't move my arm for a couple
of weeks and that was really frustrating
in a way because I couldn't interact
with my environment so this becomes a
disability yet I was feeling super fit
and I wanted to work and this is how I
discovered the Mac dictation keyboard
tool now how many of you are you
familiar with the dictation tool a few
of them do you like it yes mixed
emotions well I love it I recognize that
there is so much potential in in the web
speech API it allows to control web
applications through voice and we can
open through that for example our
applications to a lot of other people we
can be inclusive towards people with
vision impairments we can be inclusive
towards people with literacy problems or
maybe with up with the disability with
with a permanent disability if you have
heard of it before it has been around
since 2014 and it has been big natively
into the web and we will be covering
how to include voice in a demo in a
small application I wrote for this
example and considering accessibility
and inclusion as the core concerns of
the application and what this little toy
application does is it pretty much it's
JavaScript application and gets all of
my Spotify playlists so I can navigate
through them play tracks or or manage my
devices through the Kinect Web API from
Spotify which is pretty neat and to
include it the speech recognition
service works pretty much I have a very
funky looking microphone that you will
see you soon it receives sound through
your devices microphone and it checks it
against the list of provided grammars
and that's basically what we need to get
that running now a word of caution is
that the speech recognition involves a
server-based
recognition engine meaning that audio
will be sent to a web service for
processing so this is important to have
in mind because it would only work if
we're online and for that reason I have
a hotspot open because maybe our Wi-Fi
might not be that good so this is the
initial configuration very
straightforward we just instead make an
instance of the speech recognition
controller we configure the max
alternatives and the interesting thing
about it per default is actually one but
the interesting thing about it it let us
define how many words or just one word
or the list an array of words for the
service to give us back for us to work
with it so it gives us a lot of
flexibility with the kind of stuff we
can do we have higher property interim
results which will give us the the
results that are not final yet now like
I mentioned ok lack property is really
important
so if you don't define a length property
it will use the HTML tag attribute as
default or as a last default would be
the user agent language settings and as
I mentioned grammar is going to be one
of the most important aspects of this
implementation and the speech grammar
list represents an object of grammars
and uterine cies and patterns for the
service to understand so looking back to
that instantiation function you will see
that we will pass those uterine psa's
which means spoken words necessary for
the recognition I am using rxjs
observables we will not cover them in
detail but it helps me a lot with a lot
of data streams that we're handling and
the difference is that instead of
receiving them all at once we can
receive them over time so it is very
useful for this for this purpose and we
will subscribe to it and pass to the app
from string method the grammars and a
confidence score right now is just
statically one but as you can imagine
the can we can tell by every search how
important is this search for us and at
the end we'll pass it to the speech
recognition itself excuse me
that's very hot up here
so I mentioned grammars are super
important and for that we need to take a
look at NLP an NLP or natural language
processing is the field that focuses in
the interaction between human language
and computers so what it basically does
is computers try to understand process
and derive meaning from human language
and this is basically what the we the
web speech recognition is all about it
is an abstraction that takes all that
complexity away for us in order to
include speech recognition into our
application there is though an
underlying problem and human language is
ambiguous a word can have many meanings
and this is what makes natural language
processing so hard to get right
now I'm telling you from experience the
German Alexa and me we have the worst
relationship ever like I talked to her
she understands pretty much anything
even though I think I can talk German
English is better but I realized my
mother tongue is Spanish is the best and
if we're talking about accessibility
it's not going to be useful for us if
the application don't understand our
intentions and for this demo application
I thought
Alexis intent would be a really good
source of inspiration for tackling to
some extent the problem of ambiguous
language meaning you saw on the app we
can go through the playlist so there are
many ways that I can express my
intention I could say show the playlist
go to playlist we can even go further
and consider what if the service these
are actual responses I got from the
service before it says ok lists so
instead of frustrating the user I can
already consider these cases and trigger
the relator command to it
so this is a very basic example of how
something like that looks it's not even
that clean because I'm mixing languages
like you can do it way better but it
works so basically all the intentions
for me to navigate to the playlist it
will trigger one action we will iterate
over this in ten samples and we will
register to a command Factory and what
this command factory does it
administrates it manage it matches the
commands with intents we go over it one
more time and we have this callable
intent that we will need later when we
receive the result and I have a little
demo for you and I heard from Raul
yesterday I love that he said let's hope
the demo gods make this possible and I
hope that too so basically check that
I have this funny-looking microphone but
it will make it better and exploding so
it's working fine like I said Internet
is very slow I'm gonna with the hotspot
but we'll give it a try
so we would say something like show
playlist
go to playlist so we'll go to the
playlist is a very basic example then
you will see don't judge me for my pal a
list so this is basically all of my
Spotify playlists they have gone until
now and we can check this one riding the
Lightning go to riding the Lightning so
it goes actually way faster but I like I
said it's pretty slow and these are all
the tracks so I could say something like
let's try something hard like
peaseblossom boogie all right play
peaseblossom boogie
okay they better work
the API don't wants to give the right
response oh one more time
let me try something first is this
working
yeah this is working so I'll go back and
we try it one more time play
peaseblossom boogie so the response to
came and now we're playing this on so
it's pretty fun if you like vintage rock
music and and before we get done there's
one more thing I want to show you
because this is not just for English so
you have to correct me if I'm wrong I
think it is the bcp 47 tack right this
time that you can use so basically any
language supported by this tack can be
used as a recognition engine and I was
asking in Norway because I didn't knew
in Norway there's apparently like four
dialects or of Norwegian well they all
of them work and I would like to show
you how can we do something like
languages
and we will try it in Spanish and now
you have to excuse me that's not really
empathetic for me but I was taught
Spanish for a second not just for the
example and we will do something like
you see the second one vibra tropical
and what I love about it is I don't even
have to create a new instance so it
denying weekly changes just by changing
the the right property and we can try
second one talk on the side of Yang
yeah that's wrong talking with salary
yang and that's wrong again one more
time
toka masala yang
Doka masala yang hallelujah so thank you
so we have three minutes I have to hurry
up right so I I had a fallback in case
it didn't work so we can skip that and
so we covered the basics and there are
some methods that are very useful I just
want to give you the whole overview of
what you need to get this thing running
we have three methods the start method
triggers the speech recognition engine
and this case was kind of weird because
I have to press the mic button there is
for sure a better way to trigger that
but that was our case the the start
method there are the stop and a board
method which basically tells the service
stop recognizing and give me or give me
not the process results until then we
have some events too we have on error
that could be very useful if we have a
good error strategy going on because
it's really granular the letters know
what the service didn't work and pretty
much this is the result we will get it
is the speech recognition result which
is within the second area which is the
speech recognition alternative and it
will give us the list of words and a
confidence score we filter those events
and keep them to the command factory
that we saw already and it will match
those commands with the results and
before we wrap it up I hope you have had
an idea of how all the things you can do
this is just a very little example but
but I hope that you understand there are
so many Web API s and we can use them in
creative ways to actually give more
possibilities people to interact with
our applications now you might be
wondering browser support it looks
horrible
as you see it is still an experimental
API I was surprised that it is coming in
edge but there are a lot of third-party
JavaScript API s that you can use
instead for a production project there
is natural language processing from
Watson there is speech to text so
there's a bunch of libraries or engines
that you can use for that so it
shouldn't stop you from trying it and
this small project is on get up you can
try it out you can play with it you can
break it you're more than welcome to do
that and if you have any questions I
will be hanging out the next two days
and you can ask anything you would like
and that being said thank you very much
[Applause]