Next, we'll talk with Dr. Juergen Fritsch.
After earning a Ph.D degree at Carnegie-Mellon University,
he was a co-founder of M*Modal.
Today, he's the company's chief scientist.
Juergen, thank you very much for being with us today.
>> Thanks for having me.
>> You are the chief technology officer of M*Modal, I believe.
>> Chief scientist, actually.
>> And the company which, as you know,
I've been interested in for years, was founded in 2001.
But, it was quite a number of years before the technology was built into
electronic medical records systems for direct use by physicians.
Could you explain why that happened that way?
And more specifically, how you went about training the system?
>> Yes. So
this was actually very intentional on our part.
When we founded the company, we were looking for
an application of speech technologies that would tolerate the error rate
that's still present in today's technology.
If you're familiar with telephony systems and other speech enabled
dialogue systems, you're familiar with how annoying it can be to an end user to
have to deal with misinterpretations and errors by the technology.
So instead of doing that and
going straight to the end users, we decided to go a different path.
And we used the technology on the back end first.
And we did this by enabling an existing workflow,
based on dictation and transcription.
So in the US, there is billions of lines of documentation created in healthcare
every day or every year through dictation and transcription workflows.
And we enable those dictation and transcription workflows, specifically
the transcriptionists that were doing that, to be much more productive.
In that we produced a draft document that they can review and
correct, and thereby be twice as fast as if they had to type it all from scratch.
So that was an immediate use case for the technology.
In other words, an ROI attached to it and
it allowed us to build up a business.
And at the same time, collect huge amounts of data.
We collected billions of audio recordings and got them transcribed,
corrected if you will, by these transcriptionist as part of the process.
So it was a fairly genius way to basically collect the data, including
correct the transcripts, to then train the system to get better and better.
And after doing this for five, six, seven years,
we got it to the point where it was so
accurate that we could enable end users to actually use it in the front end.
And use it for the biggest paycheck commission.
So, when I first became aware of M*Modal and
traveled to your office in Pittsburgh to see it.
One of the most striking aspects of the system to me was it's ability to
go into the text that it had created from the verbal transcriptions.
Find clinical concepts, and encode them into SNOMED.
Now, the students are familiar with SNOMED as we have this interview.
They know how complicated it is, or at least they have an idea of that.
So how hard was it to actually do that?
How technically challenging?
>> It was fairly challenging.
Apart from the usual challenges with any natural language processing system.
We had to deal with the complexities of the ontology, of the SNOMED ontology,
and with the medical terminology to begin with.
So, in addition to dealing with things like syntax, semantics,
pragmatics, and just the context of the entire dialog happening.
We had to also deal with similar words, words that have different meanings.
Or, it can have different meanings in different contexts,
things like that in healthcare.
Just a simple term like cold, it can have like three different meanings.
It can be abbreviated COLD and it's a disease.
It could be the common cold, or it just could be feeling cold.
So there is all sorts of disambiguation that needed to happen.
And needed to be accurate in order to codify
these terms correctly to their SNOMED codes.
>> So how accurate is it at this point?
>> That's a tough question to answer because it really depends a lot on
the use case.
We are not attempting to codify to the entirety of SNOMED,
to the hundreds of thousands or 300,000 concepts in SNOMED.
But, we're focusing on specific subsets based on a use case.
So one of the use cases we're pursuing is chronic medical conditions.
Things like diabetes or heart diseases.
And in that realm, if you focus on a particular subset of diseases and
remodeling those with a data model that's not just including the disease.
But, also the temporality and the specificity,
and who is concerned with that disease.
You can be extremely accurate.
You can be up in the high 90s in discovering and
detecting these things in all sorts of contexts.
But if you would attempt to codify each and every concept in the SNOMED corpus
and the SNOMED ontology, you would be ending up in a much lower accuracy rate.
And it's just not happening today,
that people would automatically codify to the entirety of SNOMED.
I have not seen anybody doing that accurately.
And I think you really don't need to, because at the end,
you're always pursuing a particular use case.
You're always trying to solve a particular problem.
And for that, you really only need a subset of that corpus.
>> So what can you tell us about the technology that's under the surface?
What languages and databases, and approaches are you using?
>> So let's start on the speech side.
As your students might know, all the speech recognition, speech understanding
systems that are out there today are based on statistical algorithms.
So based on machine learning,
statistically-trained machine learning algorithms.
And there's no difference here in our company.
We do use those, a variety of those,
starting with hidden Markov models, neural networks, regression models.
Even the more complex machine learning algorithms that are on the market today,
or that have been used.
But we combine them in a fairly unique way, in that we're not just seeing it
as a pattern matching problem as most speech recognition systems do.
But, we combine speech technologies with natural language processing
technologies.
The theory is that if you get a little bit of an understanding of
what you're trying to recognize,
you can do better at the acoustic identification of the sounds.
And vice versa, right?
So it's a constraint satisfaction problem that you're solving.
And the more knowledge sources you bring in, the more accurate you can be.
So we do combine syntactic power servers and annotators, and cross taggers with
the actual hidden Markov models new and networks that do the speech recognition.
And altogether by combing them, achieve better results that way.
When it comes to identifying concepts and
tagging them in a medical corpus such as SNOMED, we use a lot of classifiers.
We use anything from a statistical classifier such as even the hidden
Markov model, all the way to maximum entropy models and things like that.
>> I know you've begun to incorporate M*Modal into commercial EMRs,
I believe Greenway here locally was the first.
How's that going?
>> It's going really well.
Initially, there was a little bit of a reluctance by the EMR community,
I would say, to even incorporate speech technologies into their systems.
There was this perception that physicians will eventually converge
on structural data entry, and will accept the drop down menus and
the choice lists that they have in their systems.
But, the opposite did happen.
The physicians were pushing back harder and harder, and
saying we need to preserve the narrative.
Not just from a productivity perspective, they
don't just want to be more productive and faster in entering the data.
But, they also want to tell a story.
They want to be able to explain what they thought about,
what the thought process was in the diagnosis.
Why they tried certain things, and why they worked or didn't work.
And that, you just simply cannot do by picking from a choice list.
You have to be able to narrate a paragraph or two.
And so, this need for, not just speech technologies, but just a narrative.
A way to answer a narrative, whether it's typing, speech recognition, or
other ways, was really something that they pushed hard for.
The EMR community has come around.
We're not aware of a single big vendor, whether it's Epic,
Cerner, Greenway Athena, or Allscripts.
All these companies have come around, and are now heavily investing into
preserving a narrative part of the chart of the physician's documentation.
And so as part of that, they are all including
some sort of a speech technology for authoring these things.
But also, they are more and
more incorporating natural language processing technologies in order
to search the narrative and find the relevant information.
And combine it with the structure data that they have.
>> When you look forward, beyond where we are today,
at the future of healthcare, and the potential role and
future of voice recognition technologies such as yours.
What do you see?
>> I wouldn't limit it to just the speech technologies.
I think for me, again, it's the narrative.
And what I find exciting in healthcare is that there is this
renaissance almost, of the narrative in physician documentation.
It started out with that.
Since 100 years ago, physicians have been writing and scribbling.
And really telling a story about their patients.
And then, it was kind of on the verge of being abandoned with all these
initial EMR systems, using drop down menus, pick lists.
And now, you see the opposite happening.
Again, physicians are creating much richer storyline and
narrative around their patients.
>> And the more these technologies mature, the more,
the better the speech recognition gets,
the better the natural language processing taggers and identifiers get.
The more use cases you see appearing.
And to me, the exciting thing is that we can get to real evidence-based
medicine in the next few years.
Where you can use these millions and millions of
clinical documents that the physicians are creating in any given hospital.
That we can use those.
That we can scan those.
That we can find trends.
That we can tell exactly for this patient of that gender, of that age,
with that disease and that outcome.
This medication worked or this medication didn't work, and
in what context.
And we can use that information to derive better treatments,
and more accurate diagnoses for other patients.
And I think that's when you really have reached that evidence-based medicine
realm that we're all talking about, and really haven't achieved yet.
>> Yeah, I think there's a very exciting landscape that you portray.
I hope the students remember these comments when in Lesson 1.
We look at some of the contemporary research efforts in things like clinical
student support.
>> So thank you very much.
>> Pleasure.
>> I'm thrilled to see how far you guys have come since I first visited you,
I don't remember how many years ago.
And perhaps, we can do this interview again in two or
three years, and see what you've done then.
>> Any time, my pleasure.
>> Thank you.
>> Thank you, Mark.