My name is Jack Simms.
I was a high school
art teacher,
moved to the desert, built a studio,
my midlife crisis sports car
is this. First sort
of documented damage to
my knee was in middle school
and I developed the disease
of the growing end of the bone.
So that, coupled with just
being very active in sports,
my left knee started
acting up. By my early sixties,
it was becoming a problem where some days
it was hard to walk around
without limping on it.
When I went to the orthopedic surgeon,
and they talked about knee replacements,
I was actually thrilled
that there was a
possible way of fixing it
once and for all and alleviate it.
There are approximately
1 million knee replacement
surgery procedures
performed in this country every year.
It's predicted that 50% of adults
will have advanced knee
arthritis, requiring knee arthroplasty
at some point in their lifetime,
and that that number of total
joint replacements
performed in this country
will grow to almost three
and a half million over the next decade.
While total knee
replacement is considered
one of the most successful
procedures done.
There are some challenges
that we still need to solve.
One is that somewhere between
10% and 20% of patients
still have problems after surgery,
but we have a hard time
measuring those outcomes.
We have subjective measures:
how patients feel,
and then we have traditionally
seeing patients in the clinic,
but that's twice, sometimes three times
after surgery over the course of a year.
What we've never had before
is the ability to track
how people do over time
on a continuous basis.
My recovery was longer
than other people's recovery.
I didn't really feel like
I had a partner in that.
We really want to be studying
those people who are at risk
for having a poor outcome.
The UCSF Human Performance Center
can evaluate patients.
We can reserve the resources
like the 3D Gait Biomechanics Lab
for those individuals that most need it.
However, access to that is quite limited.
Generally, only people
who enroll in research
studies get to experience
3-D biomechanics testing.
A sort of standard equipment
in our laboratory is Motion-Capture data.
We put markers on specific
areas on people's body,
and then as they walk
within the laboratory,
the cameras can track those markers.
We create a 3D model
which tells us exactly
where the joint locations
are. Measuring these key variables
in the laboratory is great.
However, it's not enough.
What we really need to know
is what people are doing
in the real world.
We're not the first
looking at the opportunity to use sensors
to recreate a gait lab-like experience.
We can start tracking
how people do, but not
have them come back
into an academic environment
or a lab environment to do so.
I actually had the
opportunity to attend TED
and Ivan was showcasing how the Google
Jacquard sensor could be used to interface
with their environment.
I went to Ivan to mention
that I've been doing this research
and I've been very disappointed
with the results
of using commercially available
wearable sensors
to track patients in the lab.
And he said, you know,
there's a reason for that.
You're simply collecting the wrong data,
but we can help you with that.
My first reaction was
we can do so much more
with modern Artificial Intelligence
techniques and Machine
Learning techniques. Solving problems
such as the one we're working on with Dr. Bini
requires to have a broad variety
of perspectives and broad
variety of expertise.
And I'm leading an innovation group
at the Advanced Technology
and Projects division at Google,
also known as ATAP.
Our team is highly interdisciplinary.
We have hardware engineers,
we have processing experts
we have Machine Learning experts,
and we have interaction designers.
And by working together,
we can approach this
program holistically.
The modern methods
of Artificial Intelligence, particularly
in real time and on device
machine intelligence,
which allow us to run a very complex
machine learning models which understand
human emotions on a very inexpensive
and very easily accessible
computing hardware,
such as the one we're using
in a Google Tag.
Tag is a small computing module
which is designed to go anywhere
we want to put computing in.
So the goal of this early R&D
project is to see if we can replicate
the results of the gait lab
using the Google tags. If we can do this,
this means that we can replicate
the same type of analysis in the lab
outside in the real world.
So one of the big technical
challenges here is
how can we replicate
the output of the UCSF gait lab
analysis system with a Google Tag?
So to achieve that, we use our custom
neural network architecture.
This allows us to take our raw time
series data and actually be able
to estimate directly from this in blue
what the knee angle could be.
What the A.I. team at ATAP
was able to do
is match our 3D biomechanics data
to within one degree
throughout the entire gait profile.
I didn't really think
that was going to be possible.
One of the key
things that we've learned
is that with a single sensor alone,
we were able to accurately
replicate the output
for some key variables, like knee
angular velocity.
If we are successful,
this techniques will have massive impact
on how people recover
after complex operations,
particularly with
orthopedic environment.
This idea of collecting
data real time 24/7 during recovery phase
and using that information
to improve the recovery process,
we're on the cusp of a revolution
about how we think about patient care.