﻿Thanks for having me here today. I am going to be speaking today about practicing medicine in a more
precise way, with all the trillions of points of data we all have in our electronic health
record systems, and from increasingly public data sources, including clinical trials data.
But first, I am a medical doctor, so I have to disclose my conflicts of interest. I have just
a few, or perhaps more than just a few. Suffice it to say, I have started a bunch of companies;
I consult for a bunch of companies. You may not want to believe (audio skip) data,
artificial intelligence, machine learning. In fact, The Economist, at the bottom, covers
it again and again. On the bottom right, you see that the magazine actually equates data to oil,
showing a bunch of oil rigs there with Google and other companies' logos. Many people actually say
that data is the new oil. I actually hate that saying, because oil, if you think about it,
is really a possessive type of object. Either I have that barrel of oil, or you have that barrel.
We cannot both have the same barrel of oil. But data is very different. I can have data
and create something interesting with it. You can have that same data and create something else with
it. So data is divisible like (audio skip). And we can both own it and create different things.
So the thing I like is that data is the new soil; we use it to plant ideas and watch them grow. But
it is certainly a time to be thinking about data. Now of course, the FDA has been all over this and
using data. And companies will be using data to teach computers and presenting those as
products to the FDA. And you can see, this does not go back very far, if we thinking just 2017.
Now fast forward to the middle of 2019, PMF put this figure together, which is beautiful, and you
can see 45 different products that were approved by the FDA already. And you can see the different
areas of medicine on the right, and you can see the timeline on the left. Now, even since 2019,
even during a pandemic, this has absolutely exploded. In fact, if you look just in radiology,
over 100 approved just there. This is the link at the bottom there, pointing to the
American College of Radiology's FDA cleared AI algorithms page. And this continues to increase;
I think 75 approved during the pandemic. It is an amazing time for innovators to come up
with these algorithms. Because the FDA really gives guidance on how to get these reviewed.
Why is AI back in biomedicine at all? Well, we have great hardware,
especially from the video gaming industry. It turns out those chips that move bits onto the
screen, as fast as possible for video games, are also great at being used for training
machine learning, especially deep learning. We have software libraries, like I said, that
are increasingly free, and they are from great talented teams, whether they're from Google,
or from Berkeley, or many others, and they are so available that even high school students know to
use these. There are great online groups, and even YouTube videos teaching you how to use these.
And then, of course we have enormous big data sets to work with and to train with. And the molecular
data is one of them, genomics on more people, more cases. We're measuring more things on more people
in biomedicine. And a lot of that data is public data, which is important, and I'm getting to
get in a moment. But the other big data sets are electronic health records. I'm going to spend most
of my talk, talking about one and then the other. And then, of course, we still have these many
unsolved problems that still need answers. How do we predict what is going to happen next for
patients? How do we get to drugs in a more efficient way? And how do we model the entire
population to know what to do next with them? Of course, that was a big problem during COVID.
Now let's talk about molecular day too. I'm really proud to really run this repository called Import.
We run this for the National Institute for Allergy Infectious Disease or NIAID.
We run this at UCSF in incredibly close partnership with Northrop Grumman. They run all
the technical side, we run the scientific side. Now, I think over 500 clinical studies, including
many on COVID, for these are clinical studies, clinical trials to me the next big open data
are all those clinical studies and clinical trials data.
Even we and many others have shown how this data can be used for drug repositioning,
new patient subsets, digital comparative effectiveness, and all of this data is,
immunology, or immunology related. So, it can be transplants, infectious diseases, and the like.
Yeah, here. 482 studies as of February 2021. I think it has increased in the last few weeks. Now,
nearly 70,000 individual subjects, covering 114 different diseases, including vaccine response,
immune infection responses, transplantation, preterm birth, and its complications,
and even some cancer immunology as well. You can see it is not just genetics or genomics;
it is a lot of flow cytometry HLA results. Everything you would probably want to study
as an immunologist. All that data is available and important. Freely available, too. Just sign
up with a free user ID and password. So, what have we done with some of
this data? Lots of different studies people contribute and use for many things. One particular
study that I am proud of is called The ten thousand immunomes project.
We saw so many clinical studies and clinical trials being deposited into
Import that we realized all of those control group individuals a control group means times
zero treated with a vehicle, you know, in life we call that wild type. These are all humans.
We realized we had more than ten thousand subjects that were in the control groups.
And those ten thousand subjects were measured with all sorts of different measurement modalities,
including PhiTOP, flow cytometry, immunokine, gene expression microwave and so much more. So, what
we did is we gathered all of those ten thousand and more individuals, and came up with a way to
harmonize all the measurements across each other. So, just because this gene expression, that gene
expression were measured in two different studies, we were able to use some normalization techniques
to actually get those measurements to resolve. And that is what we released as The ten thousand
immunomes project, Kelly Sanchita (phonetic) as well as many others at Northrop Grumman
and at UCSF, played a large role in putting this together. But, obviously, we could not have done
this without the contributory efforts of all the original data investigators here. That's the
power of open data like this and this and much more is available for us to use for research,
for product development, speeding up companies, and of course, AI and machine learning.
I am going to spend the rest of my time talking about the other big data to me electronic
health record data. Boy, the United States is spending billions of dollars on this kind of data.
You see three different articles here. The price tag can range from one billion to four
billion dollars for this system. Unbelievable. In fact, I'd go out and go so far as to say that
I believe now, this electronic health record data is the most expensive data in America,
because we are paying doctors to type a lot of this in. So, the narrative I want to kind of leave
behind all the time, is that I believe it will be a national tragedy if we do not use this data
to improve the practice of medicine. Of course we have to use it safely, respectfully, responsibly.
But, if we do not use this data to prove the practice of medicine, what an unbelievable tragedy
it will be. And that is what we have been working on at UCSF and the University of California.
So, with that, let me reintroduce the University of California to you. You see, it is huge. We
have ten campuses, three national labs, including Lawrence Livermore, which I think has the number
one supercomputer in the country, right now. We have 200,000 employees, a 250,000 students a year.
And within that entire enterprise, we have UCHealth, the umbrella across 20 health
professional schools, six of which are medical schools, but we have also nursing, pharmacy,
dental, public health schools, veterinary schools, that makes up 20. We train half the medical
students or residents in California, which is an enormous state. We received two billion in NIH
funding, and we have received 13 billion just seeing patients, clinical operating revenue.
Five thousand doctors get a paycheck from us every month. But 100,000 doctors write
orders on our patients that includes all the residents and fellows and all the rest.
Twelve thousand nurses care for our patients and we are not just random centers; UCSF and UCLA at
least are in the US news top ten. Of course we only mention US news when we are in the top ten.
And within that mix, we have five independent NCI comprehensive cancer centers, five NIH Clinical
Translational Science Awards. And now, over all of that, we have UCHealth, which is the
umbrella across all of the health systems here. And what we decided to do was to create a central
data warehouse of all the clinical data from each of the six academic health centers. Do you see the
logos at the bottom? You see, so San Diego, Irvine, LA, San Francisco and Davis are the
major centers, massive centers. Riverside is tiny. It's a brand new medical school, but some clinics
there, they don't really have hospitals yet, but we're going to count them just the same. But then,
once a month, they send all that data up to the central data warehouse. Now, of course,
we were happy with once a month until COVID, where now, much of this data comes once a day,
so every night, especially all the COVID data. Here's a complicated slide. Here's some of the
statistics that we have been carrying for 15 million patients over the last 15 years. It's
a nice round number, right? That's 5% of the entire United States population. But of course,
we have some data on all 15 million, but let's just count when we actually deployed Epic. So,
Epic was installed at the beginning of January 2012. If we count from January 2012 forward,
we have just shy of 7 million patients that we have treated for the past nine years. All
of this data is in our data model called OMOP Observational Medical Outcomes Partnership, of
course, the FDA and Columbia University have been talking and promoting OMOP for more than 15 years.
We love it, as a vendor neutral standard as a data backend. You can see some of the numbers 205
million encounters, half a billion procedures. My favorite, 747 million medication orders,
680 million diagnosis codes and 2 billion blood test results, or vital signs.
We have, what I love to say, everything from Tylenol to CAR T cells. Tylenol is probably
the cheapest thing we do to a patient. We give a tablet of Tylenol (acetaminophen)
and CAR T cells is probably the most expensive thing we can do to a patient. Take out some
of their T cells, train them to fight cancer, put them back and watch the cancer disappear.
Of course we have everything in between. We also have all this harmonize and match to in
the same data warehouse with California OSHPD data, which is our state regulatory data,
pathology and radiology text elements, so we are increasingly now moving to getting
all text documents. And we also can resolve a lot of this with the California death index.
This is what the dashboard looks like on the nearly million patients that have actually had a
physical encounter since January 2012 not just a blood test for example.
And you can see each dot there represents the zip code, the pie chart represents which
campus is caring for them. Of course, we have, age, race, gender, ethnicity,
of course. We can track which ones are primary care and specialty care at the bottom right,
but we have also now geolocated each of our patients to a nine digit ZIP code. And with that,
we map them to what's called the area deprivation index. This is one of our first and not our last.
One first way that we can start to look at social determinants of health, by looking at
the neighborhood characteristics of where their home address is, because of employment status,
groceries, access to food, and all the rest. One being the most favorable neighborhoods and being
the least favorable neighborhoods. Now, let's mix that back up with the Food and Drug Administration
here, right? What an amazing time it is to be a digital innovator like myself and my colleagues.
These three documents have completely changed our lives. But I'm really just going to be talking
about the middle one real world evidence. I think it's an amazing time to use clinical
data to continue to study the effectiveness of the drugs that are way post approval, not because the
FDA says we should do it or we get a grant to do it, but because it's the right thing to do.
We have to figure out are our patients benefiting from these drugs, devices, and therapies, the same
way that the original patients did in the trials? It should not be a burden for us to ask that. It
should actually be our duty and responsibility to ask those questions and that's what we do.
Just to give you an example, we found and wrote a paper, Vivek Rudrapatna and I you can see it
at the link there, Twenty one uses for real world data. This came out about a year ago, before the
pandemic in JCI lots of uses of real-world data out there, beyond just looking at effectiveness,
clinical decisions support, the fact of payers and all the rest. So, I encourage you to look
at this paper. We're proud of it and I think we're blown past 21 uses now at this point.
That just gives you some hard numbers, why I'm such a proponent of this. Here are just three
popular drugs out there, reasonably expensive, all approved, and this gives you a rough idea.
So, AbbVie obviously makes biological Humira, of course we've all heard of Humira, and some
of the original four pivotal studies are described there. The premier study which is two year double
blind multicenter study with about 800 patients, but only 500 treated with Humira in that study.
Now the drug got approved. Boom. More than 10,000 patients have been treated with Humira
in the University of California. So, this is what happens after a drug gets approved. Thousands and
thousands of patients get treated on it, and all of that data is in electronic health records now,
especially in academic level centers, like ours. It's just sitting there,
waiting for the right questions to be asked and answered and for science to happen.
Now just kind of bring it to an end here, because there's an immunology related meeting here.
Here's how we view some of that data for forward predictive and prognostic modeling
using deep learning. There's a paper that came out in JAMA Network Open, if I remember
correctly. There's a link at the bottom there. We want to predict the future state of a patient
with rheumatoid arthritis. Now, one way to track patients, staff starts with rheumatoid
arthritis is with the CVAI score, you can see, C V A I, because it seems so random, right?
Each color here is one of five different patients. Looking how many days and years
since their first score, and it seems almost random, does the patient goes up or down. Above
10 means a patient's not in great control. And you can see, even the blue patient is reasonably
calm with their disease and all of a sudden it goes up. And every other patient all over.
How on earth can we actually even think of predicting the future of a patient when it
seems so chaotic like this? Indeed we can. This is the paper, and we use deep learning to do this,
and all the clinical data, and in fact we trained on one hospital system, at UCSF. And
then tested at our safety net sister hospital, San Francisco General Hospital, the county hospital.
And what we were able to show? First of all, in fact, even though these patients are very
different, that the RLC curve so we got reasonable performance, of course, testing and training on
our own system, and our own health system at UCSF, but even when tested over at the county hospital,
we actually got reasonable performance. Now, of course that's not good enough, but what it can
mean is that that can then convince leadership there to then combine these two data sets to get
to even better outcomes, right? Because we can start with one and continually improve,
but adding on more data sets and more hospital systems, as they join. Again, all of this has
to be done safely and respectfully, keeping in mind what patients have agreed to or not,
what is actually permitted, using DNA to find data, especially through HIPAA.
Now, we got a lot of things that we still have to fix. There is bias in artificial intelligence,
of course, based on the data sets that we're using to train them, right? If you're only training,
let's say at a suburban hospital that doesn't see patients from troubling,
or challenging socioeconomic status, then that's going to be a really
negative way to use an AI algorithm, when those patients are actually coming to the hospital.
Or a better way for me to put it, is to think about it in a different domain
self driving cars. You know how important self driving cars are going to be in the future.
Would you ever buy a self driving car that only was trained how to drive in Mountain View,
California, where Google is? Of course you would never think about buying that. You would need
it tested in the snow and the rain, and all sorts of adverse conditions to know that is working,
right? And the same thing here, from medical AI, as well. Why are we so hard? Why is it so hard for
us to figure out which patients are actually being used to train these algorithms? And to go get all
the patients that we need, right? To cover that disease and the racism, and socioeconomic size
and all different conditions to make sure we can train the best possible AI for all of us.
We believe transparent reporting is going to help all of us learn what went into the guy (phonetic)
models and algorithms. We wrote this paper in Nature of Medicine. It came out about a year
ago you can see the link at the bottom where we came up with a checklist for papers and even
for repositories. What should be disclosed when these AI algorithms are put out there? Including
what is the training group and the test group, exactly what cohort definitions and all of that?
But even on the inbound side, as we start to accept these algorithms, based on data,
into our hospital systems, we're going to need to prepare our health systems to accept
and store these technologies. Now stored, and stored ship is a really critical word here. When
a new drug is brought into a hospital system, put on formulary, it's not just left on there,
right? We have committees that are constantly looking at the use of that drug. Is it working?
Are there adverse events? Is it costing us more than we thought? Should it stay on formulary,
right? The chief pharmacy officers are responsible for a lot of that, and the Joint Commission
makes sure that this work happens. Actually, it's part of credentialing for hospitals.
So, then, why don't we have any of that set up yet
for AI and machine learning (audio skip) algorithms? Who's going to credential these?
Who's going to store them to make sure that, just because now the X ray machine is brand new,
that the machine learning wasn't just fixated on some artifact of the machine, right? Like, we have
to make sure that these algorithms continue to stay up and running, and are fair and equitable,
as they're being brought into health systems. I have to thank a lot of collaborators, especially
those at Stanford and UCSF, especially Northrop Grumman, putting together Import and NIAE and NIH.
I have to thank the incredible Universal California Health team that puts all of
our clinical data together. And I have to thank, UCSF and Priscilla Chan, Mark Zuckerberg for his
Alvin support, the Bakr Foundation for their naming gift for the Institute, and I have to
thank NIH. My lab has been blessed with 20 NIH grants over these 10 institutes of NIH,
the ones that read give me more money, the ones that wanted to give me less money, but I still
love them. Lots of different pharmaceutical companies and biotech companies and, the
Sees (phonetic) Foundation, private foundations, get us to work on their (audio skip).