[MUSIC PLAYING]
ERICA HANSON: So next up we're going to welcome the next team.
This team is Tulibot from Indonesia.
And we're going to have Anu from the Cloud AI
team joining as a co-host to ask the team a couple of questions.
I'm really excited to see the smart gloves and smart glasses
that a worked on.
So let's take a look at their demo and see what they built.
[VIDEO PLAYBACK]
- 400 million people all over the world
are suffering from disabling hearing loss.
And communication became one of the biggest problem
that they are facing.
An exclusion from communication can have a significant impact
on everyday life.
It can cause loneliness, isolation, frustration,
and discrimination.
Then what can we do?
Tulibot came as the solution.
Tulibot is integrated assisted device
created to help the communication,
especially for people with hearing loss.
Smart glasses is one of the device that
offered by Tulibot as an integrated assistive
technology that's able to transcribing
audio input into text output.
- It's very nice to meet you all.
We are from Indonesia.
And we are going to do the best for the competition.
- Here's a demonstration of how Tulibot smart glasses working.
How are you?
Nice to meet you.
This are several options in Tulibot application,
virtual reality, smart glasses, this device
are connected with Bluetooth to make sure
that the phone are connected with the smart glasses.
And then we have Live Transcribe.
Hello, nice to meet you.
Deaf people are communicating by doing sign language.
It's been the idea to us to developing smart gloves
prototypes.
These gloves are able to transcribing gesture
into audio output.
- What's is your name?
[END PLAYBACK]
ERICA HANSON: Hi everyone.
It's good to see you, team Tulibot.
How are you?
NAMIRA RIZQI ANNISA: Hi [INAUDIBLE]..
MUHAMMAD ALAN NUR: Hi, good.
EVA RAHMADANTI: Hi, I'm very good.
MUHAMMAD ALAN NUR: Yeah.
ERICA HANSON: Good, good to see you all the way from Indonesia,
one of my favorite countries, been there many times.
And we have Anu, my friends--
ANU SRIVASTAVA: Hey Erica!
ERICA HANSON: --and teammate.
Hi, it's good to see you.
ANU SRIVASTAVA: It's so great to see you.
ERICA HANSON: We're so close yet so far, so within New York.
ANU SRIVASTAVA: I know.
ERICA HANSON: So thank you so much for joining us.
You want to introduce yourself.
ANU SRIVASTAVA: Yeah, sure thing.
So hi, everyone.
My name is Anu Srivastava.
I am an engineer and developer relations for Cloud AI and ML.
I'm also based in New York.
And back when we used to be in the office,
I used to sit next to Erica.
But when I work on are some of the APIs
you might have heard, like Cloud Vision, Natural Language,
and stuff that we'll see today.
So my role is to bring the power of ML to app developers
around the world.
MUHAMMAD ALAN NUR: Great, great.
ANU SRIVASTAVA: Cool, so should we get
started with some questions?
ERICA HANSON: Yeah, let's get started
with the first question.
ANU SRIVASTAVA: So I was so excited to see
this app, especially with all of the APIs that you're using.
Can you tell me what was the inspiration behind this?
And how did you get started programming
with all the hardware?
NAMIRA RIZQI ANNISA: So I remember that day when
all of us are going to market.
And we see that the [INAUDIBLE] are there.
We understand that the waiters are
having a bit of hard time communicating, especially
with the customer.
And then we also see [INAUDIBLE] that there
are four people [INAUDIBLE] people in Indonesia
who are suffer from [INAUDIBLE] hearing loss.
And that was a very huge number.
And we are understand that deaf people are having inequality
in their life, [INAUDIBLE].
And at the moment, we are thinking like why can't we
find [INAUDIBLE] a solution that might be helpful
for deaf people, especially in [INAUDIBLE]..
And for the hardware, because all of us
are starting from [INAUDIBLE],, we
are trying to make the problems in the development.
And then we start just to develop
the hardware and the program.
ANU SRIVASTAVA: Super cool.
I had a question about the gestures.
So you had the gestures.
And then you're able to output is audio.
How did that work?
MUHAMMAD ALAN NUR: OK, I'm sorry.
OK, so there are several competent in smart
glove that we are using right now.
The first sensor is flex sensor.
That's [INAUDIBLE] in four finger and thumb.
And second is an inertial measurement unit, [INAUDIBLE],,
is an electronic device that measure and report body
[INAUDIBLE] first, and angular [INAUDIBLE]
using combination of accelerometer and gyroscope.
ANU SRIVASTAVA: Oh.
MUHAMMAD ALAN NUR: The device then turned the finger movement
into electric signal, which are assigned
to mini computer in here.
In this case, is Raspberry Pi.
The [INAUDIBLE] transmit through signal wirelessly
to a mini computer, that transferred them
into spoken word using speaker in here
at about one word per second.
Then regarding the learning step of these smart clues,
first the reader need to repeat each hand gesture, something
like 20 times, something like hello, hello, hello,
until 20 times.
Then the custom machine learning algorithm
turned this gesture into letters, number, and word
they represent.
So I think that's how these smart gloves outputs
in the spoken word audio.
ANU SRIVASTAVA: Very cool.
So what's next for Tulibot?
EVA RAHMADANTI: From the next plan,
we will develop the three things from Tulibot.
So the first, we will use copy and display
model on smart [INAUDIBLE] to predict the output.
So we will not use mirror anymore.
And it will be more comfortable to use and less heavy.
And for the smart gloves, we will
use material that are more comfortable for using
for other users.
And we already developed a new feature
of the Tulibot application, that is called [INAUDIBLE]
captioning.
This feature can help deaf people when
they are in a fairly large scope of communication, like seminar,
or something like that.
So when people are at such event,
and there is [INAUDIBLE],, they will
find it difficult to get information from the speaker.
Using the previous feature would be
difficult to use because nearby songs would
interfere with the sound from the speaker.
So we provide a group chat that accommodates
seminar participants and [INAUDIBLE] speakers.
The results of speech text from the speaker
will be put [INAUDIBLE] through each participant that
is part of the group chat.
And the deaf people can [INAUDIBLE] this information
from the speaker's live section without any disturbance.
So those are the three thing that we
will continue to [INAUDIBLE].
ERICA HANSON: Wow, that's great.
I'm excited to see what comes next.
So now we're going to take a question from the live chat.
So I've also been checking some of the questions.
The question is, how much time did it take
to make this fully operational?
NAMIRA RIZQI ANNISA: So how much time
to make these things fully operational
is we are doing several trainings.
Like, I think it's a lot.
And it's actually not 100% fully operating
because there [INAUDIBLE] so many things to do
to make these things work.
And I think for now, for the smart method,
it was 95% for the accuracy and [INAUDIBLE]..
And then for the smart gloves, I can
say that the accuracy is for about 85%.
ERICA HANSON: Wow, that's great.
Thank you so much for sharing.
And this is great.
We have to wrap up.
But I really enjoyed hearing about your project
and really enjoyed the demo.
And I like the innovation with everything.
And I'm just so impressed.
And thank you for sharing your journeys and process with us.
And thanks Anu for being my co-host for this.
Thanks.
ANU SRIVASTAVA: Yes, thank you.
MUHAMMAD ALAN NUR: Yeah, sure.
NAMIRA RIZQI ANNISA: Thanks.
NAMIRA RIZQI ANNISA: Thank you, guys.
EVA RAHMADANTI: Thank you.
ERICA HANSON: Thanks so much.
[MUSIC PLAYING]