TANVI SOMANI: And now we have the team
from Eye of God, an app featuring
an easy-to-use navigation system that
helps people with visual impairment
navigate to their destinations by themselves,
without needing the assistance of others.
The app is built with Firebase, Flutter, Google Cloud platform,
TensorFlow, and more, by Anish, Gayatri, Jatin, and Priyanka,
from the KJ Somaiya College of Engineering in India.
Welcome, team.
[MUSIC PLAYING]
SPEAKER: I go with someone else.
I generally don't go on my own to new places.
TANVI SOMANI: Eye of God is an easy-to-use navigation system
built for visually impaired people,
to help them navigate to their destination safely.
ANISH PAWAR: Hey, Siri, open Eye of God.
SIRI: Welcome to Eye of God.
Please choose a navigation mode.
ANISH PAWAR: Start outdoor navigation.
Take me to Santa Cruz.
[BEEP]
TANVI SOMANI: As the car approaches,
the system alerts the user via voice feedback,
along with vibrations on the side of the belt.
ANISH PAWAR: The belt comprises of four vibration motors,
and the motor corresponding to the region
where the obstacle lies vibrates according
to the distance from the user, given by the depth map.
SIRI: Head north toward Central Library Road.
Motorcycle, car.
ANISH PAWAR: All systems run completely
on device, which eliminates the need
of internet or Cloud-based systems for processing,
and reduces latency dramatically.
SIRI: Turn left onto Central Library Road.
TANVI SOMANI: On a turn, the system
provides voice, as well as pulsating haptic feedback
to the leftmost or the rightmost motor, depending upon the turn,
to inform the user the correct direction to turn to.
The obstacle avoidance system does not
provide voice feedback for relatively less
dangerous obstacles.
The lack of voice feedback and increased intensity
of the motor depicts the same.
ANISH PAWAR: Take me to Nike.
SIRI: Taking you to your destination.
ANISH PAWAR: Indoor navigation offers text detection along
with obstacle avoidance to guide the user
to their labeled destination.
If the label is detected in the frame,
a pulsating vibration will be produced,
while a constant vibration will be produced
on detection of other obstacles in the frame,
to avoid collision.
SIRI: You have reached your destination.
SPEAKER: This is great.
ERICA HANSON: Hi, it's so good to be here with all of you.
Great project.
Hi, Anu.
Awesome.
Well, this is a great project.
Thanks so much for being a part of the Solution Challenge.
I really enjoyed Eye of God.
Anu, I know you have a couple of questions for the team,
and also if you want to introduce yourself again.
ANU SRIVASTAVA: Yes.
Hi, everyone.
My name is Anu Srivastava.
I work in developer relations at Google
for our Cloud artificial intelligence and machine
learning team.
So we get to teach people how to use these APIs and services
so you can integrate machine learning into the applications
or services that you're building so you can better
solve your problems, which is the whole point of Solution
Challenge.
So I'm so excited to get to talk to you.
This is truly an inspirational solution.
What inspires you to tackle this,
and how did you get started?
ANISH PAWAR: So it was really difficult
for us to pick a cause and objective for this project.
We all deliberated and spent time
on picking a truly unique and globally resonating topic.
Our inclination towards the specially abled population
grew during our brainstorming session,
where we were discussing on one of my dad's projects,
which is called Drishti, which means vision, where they have
taken up a challenge on developing
a prototype for reusable lenses, which
are affordable to commoners, enabling a wider reach.
We reached, on a consensus, that all are thinking about the able
ones, but what about the ones who are differently abled?
PRIYANKA HOTCHANDANI: So we thought
of doing a deep-dive session and agreed
to collect viable data to form up a cause.
The data which we got shocked us.
India has nearly 40 million people
who are blind or visually impaired,
against a global figure of 285 million.
And in a country like ours, where
there's a massive flow of traffic in every part,
navigating busy urban landscapes can get extremely challenging.
So seeing the plight of our visually impaired stranded
on the roadside without receiving adequate attention
from people, and fueled by our desires
to build something that benefits the people of our society,
we came up with our solution, Eye of God.
So we started off by breaking down our idea
into smaller chunks, majorly being navigation and obstacle
avoidance.
The navigation part was relatively easy,
due to the existence of Google Maps API.
But the real research went into the obstacle avoidance.
So we pulled up our resources to think
of more efficient ways of giving the user feedback
of the surrounding obstacles, and that's
how we laid the foundation of our project.
ANU SRIVASTAVA: Wow.
So speaking of integrating these services into hardware,
tell us more about the process of actually working
with the hardware, and what were some of the challenges
that you encountered?
GAYATRI VIJAY PATIL: So unlike the other systems
in this domain, ours states only two inputs--
that being the camera image for obstacle avoidance and location
for turn-by-turn navigation.
So we used a single device-- that
is the user's smartphone-- for capturing the image,
processing it, providing control signals to the haptic feedback,
and to access the guidance both for navigation and turning.
The directional vibrational feedback
was provided to the user with the help of four vibration
motors, connected to the ESP 32, which
are mounted on the wristband.
The smartphone provided controlled signals
to the ESP 32 via Bluetooth Low Energy.
So that was all the hardware that we used.
JATIN NAINANI: Moving on to the challenges,
the acquisition of hardware components
was quite challenging during these unprecedented times.
Moreover, assembling the circuitry
was also very difficult, as we weren't
able to access any PCB manufacturing or fabrication
services.
During stage one, we used a Cloud backend with Flask
for image processing, but we were facing a lot of latency
issues, giving us around only [? 2-3 ?] FPS This led us
to explore and switch on to on-device implementation
of the same, where we were able to achieve around 15 FPS.
Lastly, running multiple [INAUDIBLE] models
simultaneously bottlenecked the CPU,
and this left us with very little breathing space
to incorporate certain features into our solution,
such as detecting zebra crossings
and guiding the user to walk along a straight path.
These were the most significant challenges that we faced.
ERICA HANSON: Right.
Thank you.
Thanks so much for sharing.
And we do want to bring in an audience question.
So let's pull that in now.
So thanks, Dishant, for asking.
How did you test your app in the real world?
ANISH PAWAR: So because of the pandemic,
we tested the solution on ourselves, which is basically
like testing the worst case scenario,
as we aren't going to walk without vision.
So one of the team members wore the headset as well as
this belt, which allowed us to test the solution.
So the second team member was keeping a watch
on the first one to ensure the safety.
And the tester deliberately ran into obstacles
to test our solution and check the user's input and output.
So this gave us some interesting results,
and we shared them with our user study participant, who is
a developer at Google itself.
So he shared that one of our features,
which is approaching vehicle detection,
isn't as useful in the real world, as, according to him,
his safety depends on the sense of the driver,
as well as his responsibility.
That's why we had to scrape out that feature from our project.
So testing gave us a really good insight of our project
and helped us iron out a few bugs
and features here and there.
ERICA HANSON: That's great.
And thank you for showing that to us.
That's really--
ANU SRIVASTAVA: Absolutely.
ERICA HANSON: Amazing.
Awesome.
So yeah, Anu, did you have any other final comments
or questions?
ANU SRIVASTAVA: Well, I just want to say, again,
this is truly inspiring.
And something that, in particular, I really
liked about your solution is that you're
committed to making this an affordable apparatus for anyone
to acquire.
So I think that's really admirable,
and you're able to accomplish so much
with the hardware and the services that you did use.
ERICA HANSON: Yeah, great job.
Thank you so much.
And to the audience, if this is your favorite team,
please do vote for the People's Choice Awards.
That is open throughout this whole show.
So please go to slido.com, GDSC21 is the code,
and vote for your favorite team.
So thank you all so much.
It was great learning more about your awesome project Thanks.
Bye, everyone.