>> Interviewer: Okay, so thanks again, Anthony.
Next up on our agenda is Jeff Siewerdsen.
The John C. Malone professor of biomedical engineering
and professor of computer science and radiology.
Jeff will discuss how we can combine clinical data
and imaging to build models
that predict the likelihood of good treatment outcomes.
>> Jeff: Excellent.
Thank you, Doctor Berger and Doctor Rosen
for the great introduction,
and good day everyone.
Share my screen now,
and you should be seeing a title slide now
that says imaging for precision medicine.
I'm going to focus on the topic of imaging
and primarily in the context of surgery.
And even just five years ago,
we would have spoken about this
in terms of spatial precision or millimeter precision,
making sure that surgery is delivered very accurately
to a target of interest.
But today I'm going to talk about how imaging
and in terms of defining phenotype subgroups,
individualizes surgery.
And so it's really within this context
of precision medicine.
And for the end,
I'll invite you to think about how imaging
as a form of measurement
combined with deep analytics expands that capability
beyond surgery to other areas,
including even COVID-19.
Spine surgery, as a specific example
is an extremely large market
and presents a large societal burden
and healthcare cost burden as well.
There are 3 million spinal fusions per year.
It's a $12 billion market,
and it's not only a large market,
but it's increasing by 32% in a decade overall
and even more so 73% increasing that same decade
for the more elderly
and the costs are accelerating as well
with a 150% increase in cost over that same time period.
You can see it's also a very technology rich area
of surgical specialty with surgical navigation,
guidance and robotics,
intraoperative imaging, and so forth.
Unfortunately, it's an area that's challenged
in the variability of its outcomes.
And some outcomes can be pretty poor
with somewhere between 13 to 19% of patients
receiving no benefit in pain or function
and upwards of even 25%
rate of readmission to address some form of comorbidity
or revision of this surgical product.
And so those outcomes have earned spine surgery,
some bad press.
You may have seen these types of headlines
where whether certain spinal fusion surgeries are necessary
and why so-called useless surgery.
If you think about that in terms of those fractions,
it's, you know, potentially billions of dollars
of surgery delivered without a benefit.
And so we want to address some of these large questions
through approaches that give us a better handle
on predicting the outcome of a particular patient,
providing a means of decision support to patients
and their surgeons.
Conventionally this is done by one's understanding
of what I'm referring to here as clinical
and demographic data,
which is captured within the electronic health record
and combining what we know
about individual patient's clinical
and demographic status with outcomes of physical health,
mental health, and so forth,
pain and function.
And from that predicting the outcome.
But as we saw within this sort of conventional scheme,
our ability to really understand
whether an individual patient
is a good candidate for surgery,
what the best surgery or rehabilitative pathway
for a patient is somewhat limited.
And so what we are adding to this picture
is quantitative image analysis.
And so that's illustrated here in terms of the extraction
of quantitative high level image features
from a patient's peri-operative image data.
So this is really image data that are acquired
around what patient's operative procedure already,
and from peri-operative images or diagnostic images
and so forth there were able
to extract quantitative features
that will help us understand subgroups
and predict an individual's path.
And these include some of the things shown here
where just the ability to,
first of all,
certainly identify which vertebrae
are associated with disease.
What aspects of spinal curvature global spinal alignment
are pertinent, how devices should
or were or would be delivered
in within the context of surgery.
And what's the role of paraspinal tissue morphology,
fat and muscle surrounding the spine.
Currently, I would say these are certainly incorporated
in understanding of the decision support for patients,
but are done so qualitatively.
So the ability to quantitatively
extract these kinds of image features,
combine them with what we know about a patient's clinical
and demographic status and empower some more advanced,
predictive measures, machine learning,
or deep learning or artificial intelligence systems
that are able to ingest lots of these kinds of information
about a patient and give us a prediction of outcome
and shape a decision support for the surgeon and patient.
For example, a system that could help the surgeon
and patient understand,
is this a good patient candidate for surgery?
Are there factors that are likely to improve their outcome?
I hope what you see reflected here
is what Doctor Berger referred
to as the ability to measure, right.
And Doctor Rosen mentioned imaging as an important aspect
of the measurement continuum
combined with analytics, deep analytics
and the increase in connectivity
of information within medicine.
And so all of those features are evident here
in a system for spine surgery
that we're developing,
we refer to that as spine cloud.
I'm just gonna touch on three of the important elements
of spine cloud in,
in extracting those high level features
in order to be able to scale
that kind of system rather than to tailor
and manually define all those features for an individual,
we have to be able to do things automatically.
And so I'll touch on just three
of the systems that we use to extract information
automatically from images.
And the first of course
is to be able to automatically label the vertebrae
within the patient's image.
And that's just illustrated here where a patient's
volumetric CTE is analyzed by a system
that we've developed called Ortho-3D.
It's a set of artificial neural networks,
which accomplishes the task of identifying vertebrae,
understanding what is not a vertebrae, what is a vertebrae
clustering, the detections,
and accurately labeling the vertebrae within the image.
And it, as you see here, it does so with precision
and recall around 93 or 99%.
And, it gives you the kind of variability
that's comparable to what an expert human observer
will define us as the center of this vertebrae
and is able to label it with accuracy of a few millimeters.
That's a simple example,
but it starts to feed some important systems.
For example, given those vertebrae,
we can start to understand the aspects
of global spinal alignment.
And some of the measurements of global spinal alignment
are shown here, lumbar lordosis, thoracic kyphosis,
and so forth.
And so you can imagine how those labels can start
to feed systems that automatically analyze the alignment
of the spine and extract of features
that would be conventionally measured manually.
And in fact,
the automatic measurements
are able to reproduce the expert measurements
within about 93.5% of the time.
In other words,
the automatic method agrees with about 95% of the observers
and about 93% of the time.
And a final example
is just to automatically extract information
on the surgical instrumentation itself.
First of all, where should the instrumentation go?
And so we developed a method
that takes the patient's pre-operative CT,
and automatically determines
how the devices should be placed optimally
within the vertebrae,
what the optimal size of the instrumentation should be,
and to be able to do so throughout the lumbar spine
to automatically localize instrumentation as it's delivered.
So we know what devices are being delivered
to the patient.
We're able to analyze models and accurately localize what,
where the instrumentation has truly been delivered
within a patient,
and then analyze the deviations.
For example, the deviation
between where the instrument was delivered
with where we thought
it should have been delivered optimally
and any deviation there,
and also to detect complications,
the intersection of the length of the space of the screw
with the vertebrae might tell us
that we actually breached the cortex of the vertebrae,
and that might give us an important feature
on whether that's going to cause problems for the patient.
So we pulled that together,
in a preliminary study shown here,
where we did a small study
involving 64 patients receiving 84 surgeries.
And you see
some of their clinical demographic status summarized here
and over here,
you see some of the image analytics
that were extracted from each patient's peri-operative data.
And so grouping those,
we combined these clinical features
and image quantitative image features
into a machine learning method
called a boosted decision tree.
This is basically a type of machine learning method
that takes each one of these factors in themselves
as a weak predictor
but is able to combine them in a way
where an ensemble of weak predictors
could give a strong prediction,
based on just the demographic data alone,
as you see here in this plot of the sensitivity
of the predictions
versus the one minus specificity of the predictions,
this diagonal line
basically represents the line of pure guessing.
And so in this limited study
with just these limited clinical features,
we reproduced the concept that we're not,
we're not getting a strong predictions
of patients outcomes,
but by bringing in image information
from the preoperative data,
we were able to improve the accuracy
of those predictions a bit,
upwards of 71% based on the preoperative
and intraoperative image.
So by the time we're completing the surgery
based on their intraoperative images,
we have a much better idea
of the outcome of a particular patient.
And if at three months,
and if we extend that on to 12 months,
we were doing even better
whereby combining image information
into the prediction model,
we were able to increase the accuracy of predictions
of over 80%.
The really interesting thing about spine cloud
is when you go back to the decision tree
and ask what were the features
that were shaping the prediction?
What was the importance of the features?
Then you go and analyze just the top 10,
the importance of those features as shown here,
image based features are shown in green
and clinical demographic features are shown in white.
And you see that the top nine out of 10 of the features
that were shaping the prediction
actually came from the image data
rather than the clinical demographic data.
So that suggests that we're on the right track,
but there's a lot of information
that can be extracted from the images
to improve the prediction.
So the big picture is of a decision support network
is illustrated here,
where you see the inputs to the system
or what I've described a predictive model.
This is a simple one.
And we're working on deeper neural network approaches
that we think would have higher predictive value,
and the type of clinical decision support tools
that we anticipate are shown here,
selecting candidates for surgery,
knowing who is more likely to benefit from surgery,
optimizing the treatment plans
that are delivered to those patients based on evidence
and shaping patients rehabilitating
or helping to shape their rehabilitated back plates
Just out of time.
But I wanted to invite you to think about how that,
how that concept of image analytics
and quantitative image based features
could serve other areas as well.
I talked about spine surgery,
but Todd McNutt has talked about this for some time
and how the spatial relationships of image...
Of structures and images can be used to predict outcomes
in radiation therapy,
for example, a dry mouth in head and neck surgery,
able to discriminate patients who were, will,
or will not simply experience poor outcomes
and rate of therapy based on image analytics.
And what we're looking at now
is how image based features quantitatively extracted
from a chest CT could understand the genesis
and progression and treatment response of COVID-19.
And also other disease sequelae that were...
Are almost certain to arise
in the very large population of people
who have been afflicted with this disease
and have recovered.
There are too high resolution CT systems in North America.
One of them is at Johns Hopkins university.
The other is at university of California at Davis.
And I'm just showing an example here of what the boosts
and image content looks like for these kinds of images.
And what I want you to ask yourself is,
or to imagine are the quantitative features
not only of the disease, but also the normal lung.
And if the extraction of those features
could help drive predictive models
that are more patient specific for COVID patients.
All right, I'll conclude there just to say that imaging
and image analytics
are describing a new role in precision medicine
and helping complete the continuum of measurement
that Doctor Rosen described,
and in a surgical context that moves the precision paradigm
from that of millimeters of precision
to the individualized patient specific
and more over notice
that the predictive models were explainable.
We could understand what it was about the patient
that was determining their outcome.
And therefore these systems could be actionable
because we can identify features that could be changed
and make a patient a better candidate
for surgery or improve their therapeutic trajectory.
I wanna thank the collaborators at Hopkins,
including Nick Theodore,
who is the head of neurosurgical spine, and co-directs
The Carnegie Center for Surgical Innovation with me,
as well as all the engineering students
and clinical collaborators.
Thank you very much.
>> Interviewer: Great.
Thank you, Jeff.
We're pretty much out of time.
We're a little over time.
So I'm just going to actually quickly compress two questions
from the panelists.
One is from Cindy Hoffman
and the others from Cynthia Savour
So what is the rate of use of this high level imaging
and are surgeons actively using this at the moment?
>> Thank you.
That's a great question.
And I'm happy to say that everything we've done so far
uses images that are already acquired
within patient's normal standard of care.
So a patient going through a normal diagnostic workup,
those are the kinds of images for you
to use as preoperative.
And if a patient is receiving surgery,
there's imaging acquired in the operating room.
That is a fairly mainstream
even if it's intraoperative radiography.
Those are the kinds of images that we use
as inputs to the models.
>> Interviewer: Great, thank you.