Male Speaker: And we're going to start again, moving on to the next panel whose topic is
EMR and clinical phenotyping: challenges and new opportunities.
And again, with the general motif of an eMERGE presenter or reactor and/or summary, our presenter
is Josh Denny.
Are you there, Josh?
Are you unmuted, Josh?
Josh Denny: Great.
Can you hear me now?
Male Speaker: Okay, now we can hear you.
Josh Denny: Great.
I was muted by the organizer, which, you know, you're not able to unmute yourself, I think,
at that point.
So, Brandy, why don't you just drive the slides?
And some of these slides are going to be kind of a rehash of what has already been presented,
so I'll go through them quickly, and then we can move on.
So, Brandy, why don't you go ahead and advance?
Next slide.
So, this comes from our original charter, condensed a little bit.
For our goals were to develop, validate, and implement about 27 EHR phenotypes, which would
bring the total between eMERGE-I and eMERGE-II to over 40 for genomic study across eMERGE
sites.
In addition, the total gets a little bit larger when you think of things like hemochromatosis
and other projects that have been engaged.
The model for each phenotype was that there'd be a lead site that develops it and validates
it; one to two other sites to deploy, validate, revise this algorithm with their lessons;
and then deploy it across the network.
And eMERGE-II used the model that everything used exists in genotype records as opposed
to using specific cohorts as done in eMERGE-I. And then, of course, we wanted to investigate,
you know, ways to promote privacy, promote data algorithm reuse across the network and
amongst other sites.
And then the second goal was to improve the process of EHR phenotyping.
And we had this wrinkle that we, you know, were thinking about implementation here, too,
and as we've gotten into the eMERGE PGx project, things where it might actually get into clinical
care.
Next slide.
So this just summarizes where we are now.
The three colors here represent where we are.
The -- what was originally, I think, greenish on my screen, looks a little different now,
represents phenotypes that have been done really with GWAS at various stages of completion
now.
The -- you can see the -- kind of a yellowish color, the phenotypes that are expected to
be done next, and then the others in development.
So we're making good progress on the specific eMERGE-II phenotypes.
There has been some phenotypes that have been explored in addition to this set through,
some cases, even more formal algorithms and validations that then were realized to be
not feasible.
So, part of the process, I think, is an investigation process and then revision as needed to see
which phenotypes actually can be done.
Next slide.
Overall what we've learned is there can be summarized as four important parts to a phenotype
algorithm to get an accurate case and control algorithms.
And they usually all include billing codes of some sort as a necessary but not sufficient
first step, and then validation of those billing codes with things like medication data, lab
and test result data, and then clinical note data.
And sometimes we use -- employ various degrees of natural language processing.
We may use text mining, Rex mentioned.
We've done some investigation into machine learning techniques and active learning as
well for specific phenotypes.
And some -- one of the things that's not represented by this -- some of them include temporal elements
as well.
Usually they are combinations of Boolean logics operating on these different components.
Next.
And this was already gone over by Rex, so I'll skip that.
Next slide.
And Rex and Dan both mentioned PheKB, just -- right now we have about 66 phenotypes that
are in various stages of development.
Seventy-three implementation evaluations have been published.
And one of the nice components of this is it has extended some beyond just eMERGE as
a tool for others as well.
Next slide.
And so this slide summarizes the 73 pieces of implementation data we have on PheKB right
now.
And so each X represents one site implementation data, and you can see that we've broken out
positive predictive value by primary site, and then the secondary site implementation
per case and control, and the red diamond represents the median on PPVs in each of those
buckets.
I think one of the great things that we see from this is, by and large, the algorithms
have performed well at the secondary sites as well as the primary sites, so it sort of
illustrates that transportability is possible, sometimes with variation of the algorithm,
of course.
And I highlighted there one of the outliers was drug-induced liver injury, but this highlights
that, you know, for very rare phenotypes, lower positive predictive values would be
tolerated.
And I think it's okay that this sort of rare phenotype algorithm didn't perform as well
from a positive predictive value standpoint because it's feasible to review the algorithm.
So that's one of the learning points we've had as we've gone along, is a recognition
of how to optimize PPV for a given algorithm based on its goal.
Next slide.
I wanted to illustrate where sometimes this could be challenging.
One of the network algorithms from eMERGE-I that persisted into eMERGE-II, which the new
site is running this algorithm as well, is resistant hypertension.
And, overall, the algorithm performed well, but, in some cases, the necessary data wasn't
available at a given site for the algorithm to perform well.
And in some cases, there was just the difficulty of the algorithm lead to some implementation
issues, which the initial estimates of the positive predictive value didn't bear up with
closer scrutiny, and that's why you have numbers like 95 going to 46 percent and 94 to 3 percent.
Of course we've fixed all this, but the initial runs of the algorithms did find some of that.
And in one case, the control algorithm actually wasn't able to be run at a site just due to
lack of necessary information in the EHR.
So one of things this highlights is the need for, of course, careful scrutiny of how you
implement it, but also the need to be able to evaluate your algorithm and potentially
share it in a more structured way than what we've typically done, which are use of Microsoft
Word documents and PDFs, and then ways to automatically validate things, like the data
dictionary, as we've had these same sorts of problems with that data as well.
Next slide.
This shows the PheWAS catalog website.
We've put all the results -- PheWAS results that Rex and Dan had talked about earlier
on this website.
It's something you can query, you can download data, you can graph the data, search it, that
sort of thing, and then download datasets as well.
And so it's another way we're trying to share data from eMERGE.
Next slide.
This here is the record counter.
The record counter is a tool we created at -- from the coordinating center that houses
the data from those 53,000 genotype samples in eMERGE, and it allows you to quickly query
[unintelligible] things like ICD-9 codes, PPV codes, demographics, and site information
to see whether something is possible.
That goes into filling that need.
I mentioned we'll ensure the original count to help you focus what phenotype to get.
Next.
And we're doing the same thing with SPHINX as well.
So SPHINX will also have medication data in it, or I should say it already does have medication
data in it, and this covers the data that's being generated from eMERGE PGX.
So it's 9,000 people's genotypes.
Next slide.
So, key questions for eMERGE-III.
We talked about what phenotypes to explore, how to make the process faster, better, how
we can improve accuracy and reproducibility, and how we can best leverage the unique nature
of the EMR.
Next.
And so one of the thoughts we had in our -- in our group was moving beyond just disease-gene
association to more detailed phenotypes.
And we think that might be something that we can do specifically with the longitudinal
data that we have, the deep data, and a large sample size of 350,000-plus people.
So, one of those would be less common or rare phenotypes.
Pharmacogenomics to follow up on eMERGE PGX as we start the sequence and have this data
over time.
Disease subtypes that may not be available in these large cohorts that collected prospectively.
And then longitudinal phenotypes, such as change in creatinine over time, or development
of -- progression of a disease state.
And then another option -- another idea would be phenotypes through clinical implementation
such as would be deployed in a clinical decision support system.
These phenotypes would have unique characteristics such that they could be real-time implementable.
Likely, you may want to optimize positive predictive value and sensitivity differently,
and could help iterate in that learning healthcare system.
Doing these rare phenotypes or subtypes are required for your sample sizes.
They may be harder to implement, may need manual validation, and so I think there could
be a tension between the number of things we do versus the detail that we do, and we
may want to engage in fewer phenotypes to do so.
Next slide.
One example of a rare phenotype would be adverse drug events or rare diseases.
This shows that -- there's a screenshot of a case control, GWAS on flucloxacillin, drug-induced
liver injury, where they had a highly significant signal, 51 cases, just showing that rare phenotypes
don't always have to -- mean you may have stronger effect sizes, and so you may not
need thousands of cases to find signals.
The clinical impact could be greater, and given that many of these phenotypes may be
lethal, having a prospective collection such as we have at the EMR cohort may be a good
way to capture them.
Problems could be that the GWAS data may not be detailed enough, and we may need new genotyping
or sequencing to go after people within the 350,000 people that haven't already been GWAS'd
or sequenced.
Next slide.
Male Speaker: Josh, you're at 10 minutes.
Josh Denny: Okay, I'm almost done.
So, new methods as well.
We could expand common infrastructure phenotyping; using them for CDS, clinical decision support.
Machine learning and active learning could be applied more robustly.
I think genomic methods, we've talked about GWAS, we could expand those across the network,
and looking at refining phenotype algorithms to include possible cases, and how we could
capture those and include them in our algorithms; Group Health [spelled phonetically] has done
some work on that.
Next slide.
And central resources could be expanded.
One thing I want to highlight is here is structured data dictionaries and data validation tools
would be something we'd want to coordinate, I think, to have across the network.
We wouldn't want a whole bunch of standards; we would want to have one standard as much
as possible.
And I think the next is going to be the final slide.
So this is just summarizing what those questions and what we've stated, that looking at different
ways to improve accuracy, reproducibility, makes the system faster, and maybe different
kinds of phenotypes.
And with that, I will end.