>> You're not going want to miss this episode of the AI Show.
We talk about nuance,
developing a clinical research search tool
with Azure Cognitive Search.
Make sure you tune in.
[MUSIC]
>> Hello and welcome to this episode of the AI Show.
We're talking about nuance,
developing a clinical research tool with Azure Cognitive Search.
I've got some guests.
Why don't you introduce yourself.
We'll start with you, Jamin, did I say it right?
>> You did, and that's a 50/50 shot so you did pretty good.
My name is Jamin and I am a product manager at Nuance that
oversees a clinical analytics tool
that we'll be talking about today.
>> I love it. Derek, you are up.
>> Hi there. My name is Derek Legenzoff.
I'm one of the Program Managers at
Azure Cognitive Search here at Microsoft.
>> Awesome. For those that don't know what Nuance is,
can you give us maybe a little rundown? We'll go to you, Jamin.
>> Sure. Nuance is a speech recognition software company,
and now we specialize a lot in
AI tools and gone into two main verticals,
health care and then ultimately enterprise.
>> Fantastic.
What is it that they build?
>> A whole lot of stuff. But I personally work with
a clinical analytics solution
in the diagnostic side of health care.
Diagnostics is radiology, x-rays, MRIs,
pathology, specimens, tissue samples, things like that.
In diagnostics,
80 percent of every patient's medical journey
usually begins with imaging.
That's either x-rays if you've broke an arm,
it could be a CT scans if
you're getting ready for surgery, whatever it might be.
I work primarily in that part of diagnostics healthcare.
>> That's amazing. Obviously this is the AI Show.
What does AI have to do with all of this?
>> We realized that
about 97 percent of imaging data goes unanalyzed,
meaning that it's either unstructured or it's not
something that's consumable on any type
of reporting analytics tool.
There's a huge need to be able to pull out key findings from
the unstructured data or the unreviewed data within imaging.
It could be the images,
it could be the report narrative
from the radiologist or the pathologist.
We use AI tools to identify key findings and
extract those so that they're
usable and consumable for healthcare.
>> When you say useable and consumable,
is it like there's a ton of
images and I want to be able to figure out which
images are important to diagnose
a patient or is it for something else?
>> Yeah, all of the above. It's absolutely that,
but then it's also to how do you take
the key report data from a radiologist?
Because a radiologist will just dictate what they see in
those images and then that's just a report.
You can give that report to the ordering provider
or you can give it to the patient,
but it's not really structured in any way
that allows you to do any type of analytics from it.
There's a lot of opportunity in this green space of healthcare.
>> It feels like you're saying there's a lot of data,
it's really unstructured,
and we're trying to make sense out of it.
How do you do this?
>> If you'd look at a radiology report and we
want to be able to find out a key clinical question.
Let's say let's find all the patients that
we've had in the past or current that have pneumonia.
That sounds like an easy clinical question to answer,
but it's extremely difficult to actually get
the answers for that because the data you're looking at,
like we mentioned, is unstructured,
there's no key way for me to say give me all of the pneumonia
that every patient potentially has.
I oversee a product that allows you to be able to call
out some of those key clinical questions.
I'm happy to show you demo if you're open to it.
>> Yeah, I'd love it. Because I want to back up,
what's the name of the product
and if you had an elevator pitch, what would it be?
>> The name of the product is mPower Clinical Analytics.
The really the elevator
pitches getting key data out of unstructured data.
How do you answer clinical questions that allow you to be able to
drive good patient outcomes and key insights at your institution?
>> I love it. Why don't you show us how it works.
>> Absolutely. I'm going to share my screen here in a second.
What you're looking at right now
is the mPower Clinical Analytics homepage.
When you log in, you'll be able to
see that you're able to search by a particular keyword.
We have radiology domain, pathology, clinical notes.
We can put a lot of different data into
the actual application and
then I can search by particular keywords.
I mentioned pneumonia.
Let's go ahead and just bring up pneumonia.
I'm going to look for any report
within the database that has the word pneumonia in it.
I go and I click "Search" and I'm able to then get
1,222 reports in this demo
set that have the keyword pneumonia in there.
That's interesting, but it's not super relevant yet because
it just gives you all the reports
that have the word pneumonia in there.
I want to actually drill
down and I'm going to look at just the impression section.
In a radiologist's report,
the most important section of the report is impression.
We parse out each section of the report and I
want to look for in the impression section,
any of the reports that have a keyword pneumonia in there.
I click on that and I can look at
the radiologist's report right over here.
But if you'd see it says no pneumonia right there.
Also I went 1,222 reports now down to
473 reports by just looking with the impression section.
But then it also has a lot of negative findings,
meaning that there's no evidence of
pneumonia or no pneumonia found.
If I want to answer the clinical question of
give me all the patients with pneumonia,
then I need to be able to do some additional analysis on that.
What I can go over here and click on "Actions" and Optimize,
I can then maximize my positive findings.
If I maximize my positive findings,
it removes any negation terms that are
in proximity to the word pneumonia.
Then I go down to 308 reports that have
a positive finding of pneumonia that's in there.
Now I can answer
that clinical question of give me all the patients that have
a positive finding for pneumonia
within whatever date range you want to be able to pull it.
Now that I've created that cohort of data,
I can do a lot of fun stuff with it.
My 308 reports with a positive finding of pneumonia,
I can analyze that,
click on it and say what are
some key variables within the reports?
Their patient age, their status, inpatient,
outpatient and emergency,
I'm able to look at what organization they
came from and when they were completed.
A lot of these key variables that
are good for us to be able to do some analysis are there.
I can also then export that data,
I can save that search.
I can create a data set if I want to be able to
carve that out and compare it with other datasets.
I can do something called a sequential search.
I have two different searches,
I can compare them side-by-side
and find patients that satisfy both.
This is really great for cause and effect.
If I want to say, hey, how many of my patients had
a lung biopsy and then within seven days had pneumonia?
I'd be able to do a lung biopsy search
and I'd be able to do a pneumonia positive findings search and
find those patients that were in
both search results for some because and effect.
I can also create a signal where I am
able to generate an alert every
time a new patient
or a new study comes in that satisfies that criteria.
The real power with mPower is that I'm able to
build an answer and drill down to answer a clinical question.
Then we add a lot of intelligence and
intuitive features to it that we can wrap around.
Healthcare is a huge proponent of technology,
but they oftentimes don't like building technology,
and they want to be able to consume it.
For us to be able to add a lot of buttons that
do maximize positive findings,
which are proximity searches, doing sequential searches,
comparing two searches side-by-side,
creating alerts whenever a new patient comes in,
it has been very successful.
We went from being
a smaller on-prem solution
to now we've moved to a hosted infrastructure. We've scaled.
We've had over 200 research articles
cite the mPower for their research.
We have probably about 1.2 billion reports that we're
indexing right now within the application.
We've scaled a lot and we've grown a lot.
We've been able to find a really interesting niche in the market.
>> It feels like writing something like this from
scratch would take a long time.
How long dis it take to make something like this?
>> mPower has been around for about 10 years.
Really the last four years is when we've been scaling a lot.
We actually ran into
some problems with it when it came to scale because we were just
moving so quickly and our old index
solution that we had wasn't able to
actually be a hosted product very well.
It really had a lot of complications when it came to
the size of the datasets that we were indexing.
We were looking for a solution,
and Cognitive Search was where we landed and it's
been the best decision for us for sure.
We saw huge increases in performance and
accuracy with how they stem and how they
pull out a lot of stuff within the analyzer.
I know Derek is on the Cognitive Search side,
and I thank him almost
weekly for building the solution that we're able to consume.
>> It's basically built on top of the indexing that
Azure Cognitive Search does plus the stuff that
practitioners would need in this case like dashboards, etc?
>> Yeah. We've highly customized a lot from Cognitive Search.
>> Awesome. Well, I know Derek,
you've just been watching.
If people are interested in doing this software,
can you tell us a little bit about
Azure Cognitive Search and maybe tell us how
mPower used it to empower clinicians.
>> Surely, I'd love to.
A high-level Azure Cognitive Search
is a platform as a service and essentially,
our goal is just to make it really easy for you
to be able to build out of rich search experience.
What I wanted to show you today was
a little bit of what these queries that
Jamin is sending through
the Nuance and Power App, actually looks like is.
The thing I'm most impressed with what Jamin team built you is
that their still leveraging
our search syntax to be able to
help users find that information they need.
At the end of the day, that's what search is all about,
helping that person find the right information at the right time.
In VS code here I've got some queries queued up.
As you can see, I'm just issuing some search queries
to do a search service,
I have like uses a subset of Jamin's search index here.
What I wanted you to start with
that basic keyword search that
the Jamin showed us where I'm going to search in ammonia.
I'm also going to set count equals true just
so I get the number of search results back.
Then I'm just going to select a couple of these fields so
that I don't have to pull back all the data will
make either query a bit quicker as well.
We do that and you can see we get 10 search results.
You can see in this first search results,
let's talk about viral pneumonia.
We actually have a few where the impression field is null,
so we must have gotten a match in a different field,
and the rest of them, the impression field
also talks about pneumonia.
Now to drill down like Jamin
didn't when you've shown us the mPower app,
what we can do is I'm actually going to
switch the query type to be
query type equals full so we get
access to a few extra bells and whistles.
Then I'm going to do
that same fielded search that Jamin showed us.
We say your impression call pneumonia,
what that says is we're just going to scope down
this query term to that particular field so when we run this,
we see that we've gotten rid of those null values
now and we're down to seven results.
One of the really powerful things about
fielded search is you doesn't need to apply to the whole query.
You could say you're part of the query,
I want to search in the impression field,
part of it, I want to search
in all fields, or something like that.
Now let's go one step further and show
the search where we maximize those positive findings.
What I'm going to do I'm going to just paste in
some text here and is
always hard because we type with an audience.
What I could do is I'm going to also
use this search field parametered out.
Similar to what we were doing with that fielded search.
Search field is also going to
scope our query to that particular field.
Since now we have multiple clauses it makes
more sense to move that to the search fields parameter.
But we're doing a few extra things and there's more and we're
using something called proximity search that the Jamin mentioned.
Proximity search is simply one step
above phrase search and to give you an example,
let's say you're searching for
George Washington a collection of documents.
What might happen is you might
find someone whose first name is George with
a different last name and someone whose last name is
Washington with their first name
and that's going to be a good result.
We're searching for those terms independently.
But we can use phrase search to make sure like
hey we want George and Washington to be together.
But for Nuance's use case,
phrase search would be a little bit too restrictive
because the document might not say like,
no pneumonia it says there are no signs of pneumonia.
We still want to be able to catch that as well.
That's where the proximity search comes in,
where it's almost like,
I guess a fuzzy version of phrase search where we're saying, hey,
check if these two terms occur within x characters of each other.
This case I'm saying,
I want to see if
no and pneumonia occur within 10 of characters each other.
Specifically, I want to make sure that, that doesn't happen.
I want to filter those answered results.
I'm going to find your show me
everything that says pneumonia but doesn't
have any of these terms within 10 characters of pneumonia.
Go ahead and send that request.
Now you can see we've been able to scope down the results from
10 results to five to really help that person drill
down to the actual information they needed.
The last thing I wanted to show you is how
Nuance is able to actually build those graphs and allow
you to see what were the age groups or
demographics of the people who fit in this cohort.
What we can do is we can use our facets for that.
What we'll do I'm just going to paste this in and we're
going to do two separate facets.
One we're going to use a facet for a patient's sex,
and then also for patient age.
Age is a numerical field so we
also need to say, what are the bucket?
I'm just going to split this into
age buckets like people less than 20,
people between 20 and 40,
and so on,
When I run this score, you can see
there's some extra data being returned.
Now we see of these five search results,
if three of the people were male,
two of them were female.
If we look at the age, it looks like if three of
the people were between 40 and 60.
Two of them were over 60.
Now we get all these insights
that Jamin is able to render on-screen that application.
Then users can go and even add this to filter.
Maybe you want to filter down to only
see people where they're aged
40 and 60 because maybe the patient
you're working with fits that criteria.
This is the tip of the iceberg and there's
so much you can do with the search syntax.
I think Jamin and team are doing a great job of that.
>> If I'm understanding this correctly,
mPower is putting these documents into Azure Cognitive Search.
It's getting index and then these queries you can run and
pull anything you want out of
that index or anything that it's indexing on.
Am I getting this right?
>> Exactly. Once you get the data into the search index,
there's all these different unique ways you can query
that data to find exactly what you're looking for.
>> Well, this is amazing. Where can people go to find
out more about Azure Cognitive Search?
We'll go to you, Derek. Sure,
one great starting point is aka.ms/km-solutions.
That's our knowledge-mining solution accelerator.
Essentially it's a GitHub repo that just makes it
super easy to get up and running with Azure Cognitive Search.
You can throw a few documents in
blob storage and maybe within about our having a false solution
with a web UI where you can search over
your documents and you try out some of this query syntax as well.
>> Amazing,
>> Go ahead.
>> No keep going there's probably more places to go.
>> Yeah, I'll say another good link
and we didn't talk about this too much in
this session today but if you go to aka.ms/azs-semantic,
you can learn about semantic search,
which is an exciting new area
of the product we've been working on.
We're collaborating with Bing to pull all of
their state-of-the-art models to
really help improve search relevance.
It's another way you can really get
great search results out of Azure Cognitive Search.
>> Amazing. Jamin,
where can folks go to find out more about mPower?
>> Absolutely. The same,
aka.ms/nuance-mpower is a place where
you can go to learn more about
the application and some of the use cases as well.
>> Awesome. Well, this has been super amazing.
Thank you so much for being with us, my friends.
>> Thanks for having us.
>> Also thank you so much for
watching and learning all about Nuance,
developing a clinical research search tool
with Azure Cognitive Search.
Thank you for watching and hopefully we'll see you next time.
Take care, my friends. [MUSIC]