SHAD ROI DE LA CRUZ: Next up, we have
the team from Game Your Fit, an app that
keeps track of your movements in real-time
using your smartphone's movement sensors.
One of the app's Game Mode Cardio Camera
uses Google's ML Kit AI library to detect movements
that the user makes.
The app is written in Kotlin and is
connected to a Firebase project and was developed
by Aric, Jason Jeremy, Jason Christian, and Monique
from Binus University International in Indonesia.
Welcome.
ANNOUNCER: Experience the joy in exercising.
Game Your Fit.
Do you ever feel like your body has
been sitting around all day for the past year?
2/3 of Indonesians haven't been actively exercising,
and Game Your Fit is here to help.
Pick a level.
Secure the arm strap around your left arm.
Go to GameYourFit.com using another device.
Enter the room code.
And you're all set and ready to play.
With our application, you can burn 150 calories
in under 15 minutes.
That's 60% of the recommended calorie burn by exercising.
It can improve your mood, productivity,
and, of course, your health.
We hear you.
Calibrating your phone will help improve
the quality of the sensors.
Want to work out with just one device?
Try out our brand new Camera Game mode for cardio exercises.
Two simple steps.
Just set up your phone and have fun.
Through the use of various Google services,
Game Your Fit aims to motivate you to get active again.
Your phone will track your movements
using its advanced sensors and lets you
progress through the levels.
WebRTC is used to connect and to communicate between the phone
application with the browser game at GameYourFit.com.
The Camera Game uses Google's ML Kit real-time pose detection
to detect steps and every single movement your body makes
to interact with the game.
No more accessories needed.
Start your journey now.
Make sure to push yourself daily and keep your streak going.
Download Game Your Fit now.
[MUSIC PLAYING]
ERICA HANSON: Hi.
Hi, Anu.
Hi, Team Game Your Fit.
It's so good to see all of you dialing
in all the way from Indonesia.
That's great.
Thanks for staying up late with us.
Hi, Anu.
Do you want to introduce yourself?
And I know that you have a couple questions for the team.
ANU SRIVASTAVA: My name is Anu Srivastava,
and I work in Developer Relations for Google Cloud.
I specifically-- [AUDIO OUT]---- which is why I'm so excited
to talk to the next two teams.
So we get to help people learn our APIs and products,
so they can use these tools to solve challenges.
So shall we dive into the questions?
So Game Your Fit, I know that you went through a couple
iterations of this application.
What inspires you to start, and how did your app progress?
MONIQUE SENJAYA: All right.
So I personally love doing sports,
basketball to be specific.
But we all know that, because of COVID,
sports are now super limited.
I myself don't really go out to the gym
anymore, because I'm just simply scared of COVID.
Just like everyone else, I guess.
We realized that this is a huge problem for everyone
health-wise, because people are just sitting around at home
and not moving enough.
That's why we really want to make something
that is easy to use and motivates people
to move and be active.
ARIC HERNANDO: Yeah.
Actually, I would also like to add that we actually
had several ideas, but all of them
revolves around making people do something at their home.
Because, well, in this trying times of ours,
it's kind of hard to do anything.
But we ended up with, well, gamifying workouts basically.
MONIQUE SENJAYA: Yeah.
We sort of played around with onboard sensors
for the first submission, where users can just
attach their phones to their left arm
and it will act as a controller to let them progress
through all the levels.
Basically, you move, you beat the enemy,
and that's the first version.
JASON JEREMY WIJADI: But if I remember correctly,
I think after the first submission,
we got feedback from our mentor in that area
that it could be a hassle to use two devices.
So we decided to add a new game mode called "Cardio Camera"
to allow users to just use their phone to work out.
And this Game Mode uses image recognition and AI.
MONIQUE SENJAYA: Yep, that's right.
That's our final product.
It has both game modes, so that users
can pick what they want depending on the situation.
And there you go.
That's the final version of the Game Your Fit
up for now at least.
ERICA HANSON: That's awesome.
ANU SRIVASTAVA: Wow.
That's incredible.
[CHUCKLING] And so what were the challenges actually
of working with multiple pieces of hardware,
and how was it actually using machine learning on devices?
JASON CHRISTIAN HAILIANTO: So at first,
we tried to connect the browser and the phone directly using
network sockets, which is possible
when the two devices are connected
to the same Wi-Fi connection.
However, raw TCP and UDP sockets are still
an experimental feature in most browsers,
and they straight up does not work in others.
ARIC HERNANDO: Yeah.
That, and we were also concerned with,
even if we do manage to get it working,
we're afraid there will be latency issues.
There will be bugs.
So imagine doing a push-up and then the game response
half a second later.
That would feel bad in a game, so we
decided to use WebRTC since it's faster.
It's more flexible.
And we can connect devices even when
they're on different networks.
JASON CHRISTIAN HAILIANTO: And for the AI in the second phase
of Solution Challenge, we develop Cardio Camera,
I said previously, that implements
machine learning on mobile.
Using the help of Google's ad ML Kit pose detection,
we're able to use the phone's front-facing camera
to capture live video of the user
and transform it into something like a stickman-like figure
that we can use to detect positions of each limb moving.
And with that, we can detect jogging,
the touching of certain regions on the screen, and many,
many more.
Users are able to interact with things which
appears on the phone screen, something
like a makeshift augmented reality, you might say.
And model itself was quite easy to set up from Google.
JASON JEREMY WIJADI: Yeah.
So for future plans, we're currently
in the works on the device calibration for the sensors.
Some have found it to be either too sensitive or not
sensitive enough when doing certain exercises,
and trying to standardize that when there are so many
different phone models can be quite challenging.
So probably use a calibration as our solution for that,
but the details for that are still in progress.
Thank you.
ANU SRIVASTAVA: Awesome.
ERICA HANSON: Thank you.
We do have a question from the audience.
I'm going to pull that up right now.
While I pull that up, I just want
to remind people to vote for the People's Choice Award.
If this is your favorite team, please do vote now.
So just remember to go to slido.com
and vote for your favorite team.
So let's take the audience question.
How much time does it take to train
the captured data and then process to scores of the game?
JASON CHRISTIAN HAILIANTO: So at the first iteration,
we actually have to train the detectors.
So we have to take a lot of data.
So we took data when we are recording and strapping
our phones to our left arm.
And we took a lot of data, and we tried to process it manually
at first.
And we tried to set a high threshold and a low threshold.
So, for example, when a person is doing push-ups,
it can be detected more easily and it
works for a lot of people.
I think Jason Jeremy can elaborate more on that.
JASON JEREMY WIJADI: Yeah.
So we're still using some threshold data
that we have from before.
But after this, we're still working
on that on device calibration for the exercises.
JASON CHRISTIAN HAILIANTO: So the calibration feature
actually helps everyone from all sizes and all heights
to be able to play in our game.
So they just have to do a simple calibration workout
before doing any real campaign workouts,
and the system will automatically
adjust to their custom sizes.
ERICA HANSON: Awesome.
Thank you, and thanks to the audience for asking questions.
And, yeah.
It was great learning more about your project.
I would love to use it.
I definitely need to be working out a lot more,
and I do need help.
So thank you for even thinking about this.
Anu, do you have any final thoughts or comments?
ANU SRIVASTAVA: Absolutely.
Yeah.
I can't wait to try it out.
This was super cool, and I really
love that you gamified it to help people
get excited about working out.
And the research that you did into the needs of exercise,
also very impressive.
So I would love to see how you continue
to experiment with machine learning and hardware
and see where it goes.
ERICA HANSON: Yeah, that would be great.
And Anu and I both live in New York City, so--
ANU SRIVASTAVA: Yeah.
ERICA HANSON: --we can meet up and we'll definitely
use this project.
ANU SRIVASTAVA: Let's go outside to the park.
[CHUCKLING]
ERICA HANSON: Exactly.
I like that.
Nice.
Great.
Well, it was so nice meeting you.
Game Your Fit Team, great work on this project.
And once again, to the audience, if this is your favorite team,
please vote now for the People's Choice Award.
Thanks so much, and have a good night's sleep.
Hope you get some rest tonight.
Thanks for joining.
Bye, everyone.