LAURA CINCERA: Last but certainly not least,
a huge welcome to the SimplAR team.
SimplAR is an app that utilizes the power of natural language
processing to translate any text into simplified language
only by taking a picture of it.
Flutter and Firebase were the main ingredients
used to develop SimplAR.
Let's please give a warm, warm welcome
to the creators, Almo, Maria, Sami, and Viviana,
from the Technical University of Munich in Germany.
[VIDEO PLAYBACK]
- The concept of functional literacy
is based on the UNESCO definitions which
cover a continuum of proficiency levels rather than a dichotomy.
You didn't understand the meaning of the sentence?
[MUSIC PLAYING]
Functional illiteracy, the inability
to understand texts on more than a basic level,
is the reality for one in seven people worldwide.
Among others, one common cause is intellectual disabilities.
In a world where we communicate mainly through text,
people with functional illiteracy
are excluded in many ways.
UN goal 10.3 specifically states, "to empower and promote
inclusion of all," including people with disabilities.
When talking to experts, we learned
that there are simplified language frameworks that help
with functional illiteracy.
However, these texts are translated by hand,
so only very few texts are made accessible.
- This is where SimplAR air comes in.
We are using the power of natural language processing
to automatically translate any text for functional illiterate
people.
When first opening the app, the user
is introduced into the W3C-verified
disability-friendly UI.
In our Flutter app, the user can scan any text.
Firebase ML extracts the text and sends it
to our backend hosted by Google Cloud.
The backend uses a mix of GPT-3 and our own machine
learning models to process the text into a simpler version.
The simplified text is returned to the client, which
then shows it as an AR overlay.
Some texts are so complicated that an AR
overlay would be unreadable.
In this case, we open a new screen
with the plain language text and supporting images,
as recommended by the plain language framework.
Using SimplAR, function illiterate people
can understand any text in any medium independently,
and we can have a more inclusive world.
[END PLAYBACK]
ERICA HANSON: Hi.
Hi, again, Victor Hi, team SimplAR.
So good to see you dialing in from Germany.
That's great.
Victor, do you want to introduce yourself again?
And I know that you have a couple
of questions for this team.
VICTOR GRAU SERRAT: Thanks, Erica.
Hi, team.
Hi, everyone.
I'm Victor.
I'm based out of the mountains of Colorado here in the US,
where I work remotely as Engineering and Technical Lead
at the UNICEF Office of Innovation,
where we are constantly exploring and prototyping
frontier technologies for the benefit of every child
everywhere to build a better world for everyone.
And I only bring this up because I
think that this should resonate with your participation
in this year's Solution Challenge
and the amazing application that you have built to help people
with disabilities in understanding text, which is,
I find, a very hard problem.
So I am glad that we have this very capable team tackling it.
This question all the judges have asked the other teams,
but I feel really compelled to understand this,
so I'll ask it again.
What was the biggest challenge you faced
while building your solution?
And put some color on how did you overcome it.
MARIA POSPELOVA: So yeah, let me start with the question.
So actually, the biggest challenge of our project
was the text simplification itself.
Like you already said, it's a really difficult task.
So it may seem maybe at the beginning
that it is really simple just to take a text
and just write a simple version of it
so that, for example, a two-year-old child can
understand what DNA is or something like that.
But for technology, it's really, really hard.
And this is because this problem of text simplification
is currently one of the most--
or one of the research topics in natural language processing.
And there are only a few data sets
to use to train a machine learning algorithm on
so that it really can give good results at the end, so yeah,
and especially in German.
And also, there are a lot of different possibilities on how
to approach this problem.
ALMO SUTEDJO: And so what we did is
we collaborated with many experts in natural language
processing, so for example, professors or PhD students.
And what we did is we evaluated the state of the art methods
for tech simplification.
But what we noticed is that they had their own strengths
and weaknesses in different domains.
So to improve the stability of our models,
we combined multiple machine learning methods, including
GPT-3, for example, and we created our own pipeline,
which adds additional pre- and post-processing to the text.
VICTOR GRAU SERRAT: That's great.
And thanks for the mention on GPT-3.
I just want to make a quick comment to the audience.
If you are not aware of this, it's
a natural language processing model
that is trained on 100-plus billion parameters.
So when we talk about complexity,
this is the real deal.
Switching gears, in the demo that we have just seen,
the second example simplifies legal text
for easier understanding--
I think something that we could all benefit from.
But it concerns me because it has some implications
on the legal implications of handling legal texts
and the liability of such.
So what if some important detail is lost and causes
the wrong action or inaction that negatively
affects the individual that is reading the simplified version?
Can you comment on that, please?
SAMI WIRTENSOHN: Yeah, thank you very much, [AUDIO OUT]..
First of all, we want to clarify that SimplAR is not
aiming to provide legal advice.
Instead, we are giving these people an opportunity
to be more informed about what's happening around them.
Our main use cases are everyday interactions,
like reading newspapers, restaurant menus,
or, for example, emails.
VIVIANA SUTEDJO: But what we did learn in our expert interviews
is that, for example, people with more severe cases
of cognitive disabilities have an assistant
to help them with legal matters, and we are not aiming
to replace these assistants.
But these assistants cannot always be there all the time,
so we are closing this gap between everyday life tasks
and the tasks where an assistant can help them.
VICTOR GRAU SERRAT: Got it.
Thank you for that clarification.
Much, much better now.
Back to you, Erica, for the question from the audience.
ERICA HANSON: Yeah, sounds good.
We got some questions.
So the question for all of you is,
what makes this application different or unique
from other translation apps?
ALMO SUTEDJO: Sure, [AUDIO OUT] with the answer.
So what our app does is it doesn't translate,
for example, from English to German, for example.
What it does is it simplifies the language.
So for example if there's a really complex sentence,
like for example, from Wikipedia,
and there are many people who can't
understand this, especially people with disabilities.
So what we do is we make these texts
more accessible for those people by simplifying these words,
for example, using different techniques, for example,
lexical simplification, by replacing difficult words,
or for example, restructuring the sentences into shorter
sentences, making them all more simple,
and also providing a really intuitive UI,
and providing more images for them.
So what we do is we make texts more accessible for people.
MARIA POSPELOVA: Yeah, and I also want
to add that we also mainly focus on texts in physical form,
not in internet form, but in physical form.
Like some of us already mentioned
for example, newspaper, or if you get a letter from some,
I don't know, insurance company or something.
We get it a lot in Germany.
So it's pretty easy to just take the text,
type it down somewhere in Google,
and simplify it to a simpler version.
So with our app, you can just take a photo,
and you get the simplified text on screen or with photos,
like Almo already mentioned.
ERICA HANSON: That's great, thank you.
Victor, did you have any final comments?
VICTOR GRAU SERRAT: Yeah, just building on what Maria said,
I think that we get letters from the insurance companies
everywhere in the world, unfortunately,
and they are never good news.
So I think we could all get some help with those.
Thank you very much for tackling this problem.
Jokes aside, I think it is a very important problem that
has been unaddressed for a long time,
so I'm glad that you are tackling it now.
Best of luck.
ERICA HANSON: Thanks so much.
And to the audience, if this was your favorite team,
you all have about two more minutes left to vote,
so please do vote for SimplAR or any team
that's your favorite right now at slido.com.
And we all know the code is GDSC21.
So thank you so much, SimplAR team.
So good to learn more about your project.
And thank you, Victor, good to see you.
VICTOR GRAU SERRAT: Good to see you too.
Bye.
ERICA HANSON: Bye, everyone.
Great, so thank you all so much for sticking with us
for the last couple of hours.
We do still have announcements, so do not
leave the livestream yet.
We haven't heard yet who the top three
winners are in the People's Choice Award, so get ready.
But first, what did you all think about the demos?
Add in the chat, let us know what you thought.
I loved these demos.
It was so cool to see the different bits of technology
they used, the different types of UN Sustainable Development
Goals they solved for, really inspiring.
And I'm wondering if you're thinking
about what projects you're going to work
on next year for the Solution Challenge.
A quick plug is that we will launch it again
in January of next year, so I'm excited to see what you all
demo next year.
So right now is the last opportunity
to vote for the People's Choice Award.
This is your last chance to vote for your favorite team,
so please do that right now.
You have two minutes, two minutes left.
Message your friends and say, hey,
it's time to vote for our favorite team.
Which team resonated with you?
Is there a project that inspired you?
Maybe you live in the same country
as the folks on this team.
Whatever your reasoning is, vote for your favorite.
You have two minutes left.
And you go to slido.com, enter the code GDSC21,
or you can scan the following QR code.
And vote right now.
So you have two minutes.
You're on the clock.
And what are we going to do in the next two minute?
I'll share that right now.
So before we announce the People's Choice Award
and the top three winners, we wanted
to share a few tips and best practices from the students
that you've heard from today on how to develop solutions.
So I hope that it will help you as you
work on your projects, hopefully, for the Solution
Challenge next year.
But these are some great tips, so let's play that video.
Let's check out the tips.