[MUSIC PLAYING]
MARCOS NOVAES: And now, for the "Lightning Talk" that everyone
was waiting for.
Finally, a turnkey solution for labeling medical images
and training great medical AI models.
Hello.
My name is Marcos Novaes.
And I'm a Solution Architect with the Google Cloud Health
Care and Life Sciences Teams.
And in this talk, we will discuss the challenges
in the area of medical image annotations and the impact
that medical annotations have in the development of accurate AI
models for medicine.
And to lead this discussion, it is my pleasure
to welcome today my colleague Brad
Genereaux, who is Global Lead for Health Care
Alliances with NVIDIA.
So Brad, what are the challenges related to medical image
annotation and how can we overcome them?
BRAD GENEREAUX: Thanks, Marcos.
That's a great question.
You know, one of the challenges in building
great medical imaging models is to have great medical imaging
annotation.
And to do that, we need clinical expertise,
clinicians spending their valuable time
to review medical imaging studies
and identify organs, diseases, anomalies and much more.
They need to be able to access annotation tools on demand
connected into the applications they already use
and require a seamless resilient experience.
Our domain needs SDKs to put this
within reach of our medical teams.
So let's take a look at the medical imaging AI ecosystem.
We can break down AI in the medical imaging space
into really two groups of workflows, training
and inference.
We're focusing today on that first step,
curating data from imaging sources
and creating ground truth.
Once we have that ground truth, we're
going to use that to train our AI models.
We might take existing AI models and fine tune it.
We might work together with other hospitals
and use federated learning and train models together
while protecting sensitive patient data.
Once we have a model, we would then package that up
for validation and for evaluation.
On the inference path, we're going to be taking that model
and packaging it up into an AI application.
We would take that AI application
and connect it into our medical imaging workflows from our PACs
or from our modalities sending DICOM back and getting results,
and then, ultimately, visualizing those results
in our medical imaging viewers.
But today, we're focusing on creating great annotation.
So let's take a look at our clinical scenario.
It bears repeating, a high quality AI model
starts with high quality labeled data.
So we look at our clinical scenario.
Imagine that we have a hospital, we've got clinicians,
and in this hospital, a radiologist
has been asked to annotate 100 imaging studies to identify
the kidneys in them.
So they're going to--
so our clinician here will open up an imaging study, a CT
study, navigate down to midway past the kidney,
and using the MONAI label tool, click on the kidney
and then have that selected in 3D space.
That clinician will then do a quick review,
do any minor touch-ups as necessary
and accept that annotation, and then move on
to the next imaging studies.
So the question is, how do we make this a reality?
This is where MONAI SDK comes in.
MONAI is an open source consortium,
an SDK that helps us accelerate the pace of research innovation
with a common foundation.
It's built together from academia and industry
from all around the world to take the best and brightest
into an SDK that we can take and then
integrate into the rest of our medical imaging ecosystem.
It has a number of different components following
those two groups of workflows that I discussed earlier.
We've got MONAI Label, which allows
us to connect into viewers with an SDK
to be able to annotate rapidly our medical imaging studies, so
whether it's organs or anomalies,
as I previously mentioned.
We have MONAI Core that helps us take ground truth that we've
identified and create those medical imaging models.
We then have MONAI Deploy to help
us package and connect into our medical imaging ecosystems.
Taking a look at the architecture stack,
it starts from our platform with our--
either with Google Cloud and Vertex,
with DGX, with our off-the-shelf servers that we have
in our hospital data centers.
We layer on the platform, we layer on the MONAI tooling,
and we end up creating applications
to service the needs of our clinicians
within our hospitals.
We're going to take a little bit of a deeper dive on the MONAI
Label piece, but there's components
that address everything with a shared stack
that we-- connecting to all these different applications.
So taking a look at MONAI Label in particular,
an open sourced SDK that we connect
into our medical imaging viewers.
We already done some of the heavy lifting
to connect it into open source 3D slicer
and as well as a zero-footprint web viewer called OHIF.
Doing these integrations really help developers,
really help the entire ecosystem to drive forward
connecting MONAI label to help us annotate studies.
What we're able to do with MONAI label
is send a DICOM using DICOM web to be
able to access that entire 3D space of a medical imaging
study, do the inference, that annotation,
and ship that back to the viewer for rendering.
MONAI Label even has an active learning component
and gets smarter over time so that as we
are annotating our imaging studies,
the accuracy just keeps going up.
So all of this, connected with our medical imaging viewers,
helps to enable our radiologists and clinicians as part
of their annotation process.
So looking at this, as I mentioned,
to do great annotation, we need a great SDK
and a great platform to run this on.
So Marcos, I'd love to hear from you.
From a platform perspective, where do we put MONAI Label?
MARCOS NOVAES: Well Brad, look no further.
You put it in Google Cloud, of course.
And here I am demonstrating a diagram that
shows my favorite way-- there are several configurations that
are possible, but I'm showing a very nice way
to implement MONAI Label in Google Cloud.
As I said, Brad, MONAI label supports the DICOM standard.
And here I'm demonstrating how you
can use the Google native DICOM store as the storage backend
for MONAI Label.
And that gives you a lot of scalability.
And I'm also showing here that there is an interesting option
to deploy MONAI Label within the 3D slicer application embedded
in a Jupyter notebook.
And by doing this, you will have all the power of MONAI Label
together with the power of the Python language.
And in this way, you can access the other components
that I'm showing there, which is BigQuery, cloud
storage, and most importantly, you can access and orchestrate
AI models running in Vertex AI.
And now this is accessible through a web browser,
because that's the way that Jupyter Notebooks work.
And here you can see a screenshot
of this deployment in practice.
We are looking here at the window of a Jupyter Notebook.
And within it we have this graphical application
to a remote desktop that is embedded
in the notebook which is showing the 3D slicer application.
And at the left side, you can see the MONAI Label plug-in
and you can see the active learning component there.
The user can select images to annotate those images
and submit the annotations, and that
will cause the annotations to be written back to the DICOM store
in the DICOM format and makes them immediately
visible to other researchers.
So as you can see, in the terms of the whole workflow
of developing AI models, that the notebooks take
center stage in this workflow.
And inclusive because the AI label that happens in MONAI
label is an interactive process.
So there's actually a cycle of improving
the data set as we go along.
And the notebooks allow the researchers to do this,
at the same time having access to Vertex AI
and calling in other parts of the MONAI framework as well.
So what's next?
Well, join us in the effort to build better medical AI models.
Visit MONAI.io and check out the latest features
and join the communities.
And then deploy MONAI in Google Cloud and leverage
the power of Vertex AI.
And together, we can then build better medical models
and help us to advance the science.
So Brad, thanks so much for the presentation
and for showing to us the power of the MONAI framework.
And thanks, everyone, for listening.
[MUSIC PLAYING]