[MUSIC PLAYING]
JESSICA DENE EARLY-CHA: Welcome to "Assistant On Air,"
where we have conversations with people that
build for Google Assistant.
I'm Jessica, and I'll be your host today.
And today we'll be chatting with Joe, who's a Google Developer
Expert for Google Assistant.
Welcome, Joe.
JOE BIRCH: Thank you for having me.
JESSICA DENE EARLY-CHA: Awesome.
Joe, can you tell us a little bit more about you?
JOE BIRCH: Yeah, my name's Joe, and I'm predominately
an Android engineer.
And I work remotely from the UK.
I've been doing Android for about seven to eight years now.
And here recently, I've been shifting my focus
into more technology areas to learn new things
and broaden my horizons.
JESSICA DENE EARLY-CHA: That's awesome.
I know the way I found out about you was your blog
post about Chord Assist.
Can you tell me about Chord Assist?
JOE BIRCH: Yes, definitely.
Chord Assist is an accessible guitar
that I built to be used by people who
may be blind, mute, or deaf.
And the idea behind it is to bring music to more people.
Music was always something I really
enjoyed as I was growing up.
And I would just love for more people
to be able to have access to that.
JESSICA DENE EARLY-CHA: Wow, that's awesome.
So can you tell me about the technology
that you used to make this happen?
JOE BIRCH: Yeah, so there's a bunch
of different things happening behind the scenes for Chord
Assist.
The main thing is the Assistant.
So it's using Dialogflow to power the brains of the guitar,
as I call it.
And there's also things such as Firebase Storage.
It makes use of chord diagrams, so those diagrams
are stored within Firebase Storage.
It uses the Google text-to-speech API
to handle anything that needs to be translated
to audio and speech and so on.
And it also uses a Raspberry Pi, which
is just the machine that drives everything that's inside of it.
JESSICA DENE EARLY-CHA: That's awesome.
So if I have some type of hearing capability
that's limited, I can just play the guitar,
and it will do what?
JOE BIRCH: So the idea is that when
you're learning the guitar, the key thing is chords.
So whenever people start to learn the guitar,
chords are the fundamental part of being
able to get used to strumming and picking strings
and just being able to make up sequences
if you're by yourself.
And so the main idea behind the guitar
is to teach you those chords.
So for someone who may be blind, for example,
they can speak to the guitar, and they can
request how to learn a chord.
So they could say, hey, OK, Google, tell me
how to play the C chord.
And the guitar has three different toggles.
So you can turn on the Braille, you can turn on the screen,
and you can turn on the audio.
So it provides options for everyone,
depending on where you are at the time of learning or just
depending on how you feel and what
you want turned on and off.
And if you're blind, it will use the Braille reader
on the bottom of the guitar to read out the instructions
on how to play the chord.
So it may say, E string open, A string fret one, and so on,
to guide you where to place your hands.
On the other hand, if you don't read Braille,
then it has a screen which shows you the chord diagram.
So chord diagrams do require a little bit of background
knowledge on how they look and how you read them,
but that will, essentially, display a diagram which
the user can use.
So that may help in situations where also you're deaf.
So if you're hard of hearing, you
may not be able to use the speaker.
So the speaker is used to, essentially, read out
the instructions.
It's the same as what's used for the Braille,
but just in an audio format.
So if you're deaf, then you would use the screen.
JESSICA DENE EARLY-CHA: That is amazing.
I'm not very musically inclined, so I
remember when I was reading that, I
was like, that sounds so fantastic, because it
is hard and challenging.
And you're just opening this up to a whole group
of folks that didn't have this as an option before.
JOE BIRCH: And it's interesting because it also
has a selection display.
So if you're mute, then you can select
one of the available chords to learn.
And that was one of the things I added at a later date
because as you get more involved in the accessibility kind
of projects, you start to think about different routes
and different avenues that people may
enter into the experience with.
And it just opens your eyes a bit more.
JESSICA DENE EARLY-CHA: Definitely.
So what was the inspiration of this action?
JOE BIRCH: So there's a condition
that runs in my family called retinitis pigmentosa.
And what that essentially does is over time,
the cells in your eye-- your retina--
die.
And they show up in eye scans as black dots and stuff.
And what that essentially can lead to is tunnel vision.
So my grandmother had it.
And her eyesight ended up being like a pin in her later life.
So it really does vary between people.
Some people don't have it that bad.
They can still have good eyesight
up until their lifetime.
And so my mom has it as well.
And she can't drive, but her eyesight is still good.
But she's registered blind, but she can still function.
So that was the main reason--
the main influence-- behind it.
JESSICA DENE EARLY-CHA: That's amazing.
That's amazing to be able to identify something that's
personal and to be able to build this out and share this out
with the world.
JOE BIRCH: Yeah, I definitely think having a close connection
with these kind of experiences and seeing
what people can go through really
gives you that extra space to think about these things
and give you that motivation to be like,
oh, I want to make a change there.
JESSICA DENE EARLY-CHA: Yeah.
How has the feedback been from the community
when you announced this project that you were building?
JOE BIRCH: I had some really good feedback, actually.
I kept it quite quiet.
I mentioned it once at a conference
because I kind of wanted to commit to the idea,
because it was taking me some time to build it.
So I was like, if I just tell some people about it,
it might be a bit of a push for me to get heads down on it.
So no one knew I was building it, and I shared it.
And I think it's quite unique.
I don't think anyone's-- that kind of product doesn't exist
currently.
And I think in the community, nothing
like that's been built before, and I
think there's so much that has been brought together
to build it.
I think people really liked seeing
that combination of things put together,
especially for something that's for a good cause.
And hopefully it inspires people to explore similar things
and to get outside of their comfort zones,
because to be honest, I'm not a hardware person.
I haven't done much with hardware before.
So for me it was a really good experience to learn things.
And people know I don't do much in that area,
so hopefully it's like, oh, we can do these things as well.
JESSICA DENE EARLY-CHA: It's doable.
If you could go back and talk to old Joe
and give him some words of advice, what would you give?
JOE BIRCH: Oh.
[LAUGHTER]
I think initially I would have probably done some user
research to begin with, because I think the guitar does a lot.
It's got the Braille.
It's got the screen.
It's got the audio.
It's got the piece for people who are mute.
And maybe some things might not have been needed.
Maybe the chord selection area might not
have been needed on the guitar.
That could have saved me time.
But I tried to just cover as much as I could
so I could show that things are possible.
As well, some background electronics knowledge
would have been good.
There's was a lot of soldering.
I broke a few things.
And I spent a bit more money than I hoped.
So my dad's into electronics, so I would have probably
asked him for more help.
But I asked him for some.
I remember on Christmas day, they came over for dinner.
And I was like, Dad, can we go upstairs after dinner
because I need you to show me how to do this?
But I wish I had done more of that along the way
because I'm not too strong in that area.
JESSICA DENE EARLY-CHA: Yeah, that's awesome.
Well, thank you so much, Joe, for hanging out with us
and sharing this with us, this amazing action that you built.
Really appreciate it.
JOE BIRCH: Thank you.
JESSICA DENE EARLY-CHA: Thanks.
So I hope that you liked today's show.
If you have any suggestions for future episodes,
please leave a comment below.
And follow us on Twitter for updates on new episodes.
I've been Jessica.
Chat with you next time.
[MUSIC PLAYING]