Hi, everyone. I'm Brenna, from the University of Toronto. I'm
here to present our paper, Automating Clinical
Documentation with Digital Scribes: Understanding the
Impact on Physicians.
For many of you, this scene might look familiar. Last time
you were at the clinic, you might recall seeing your doctor
busy typing away while you were trying to tell them about your
concerns. It might even seem like they're more engaged with
the computer than with you, their patient.
However, doctors are having a hard time too. The amount of
documentation that's required of them since EMRs, Electronic
Medical Record systems were introduced has gone up
significantly. EMR documentation takes up to 25% of their in
clinic time, up to 2 hours of additional after clinic time, a
d has become a leading cause f physician burnou
A work-around solution that has gained popularity is to employ
medical scribes, individuals who are trained to make EMR
documentations for physicians. The use of medical scribes have
shown improvements in efficiency and patient relationships, but
it's limited in scalability and economic costs.
This motivates many people to explore the use of technology,
which leads to the concept of digital scribes, a system that
takes patient physician conversation to generate
clinical notes during the encounter. While, this concept
appears convincing. We know very little about how digital scribes
will be used in practice.
This leads us to our research questions, which is, 1. To
understand how physicians interact with digital scribes,
especially since the current technology is still in
developing state. How would physicians perceive and react to
the system's strength and limitations? 2. To understand
how digital scribe systems can impact physicians and their
workflows?
For the study, we designed several meidcal scenarios that
were acted out by a standardized patient. We recruited 24 general
practitioners to go through each of the scenarios. During this
time, a digital scribe prototype was running live in the
background generating notes. We then presented the generated
notes to our physician participants for a think-aloud
review session and asked them to rate the note they saw, before
finally concluding with a semi structured interview. In a
figure below we have an example of the study set up. Labeled A
is an one-way mirror, which researchers sat behind to
operate the digital scribe. B is the physician participant. C is
the EMR. D is a microphone. E is the standardized patient.
The digital scribe note system would first capture the
conversation in the exam room from microphone and then
generate SOAP, which stands for, 'Subjective, Objective,
Assessment, and Plan' structured clinical notes to present to
physicians in the EMR for review. As digital scribe
systems are still in development, we designed a
prototype of 3 conditions, Machine, Hybrid, and Human to
represent the current, intermediary and futuristic
implementations of the technology. Machine scribe
conditions are purely machine based. It takes the audio data
converts that to a text-based transcript through the audio
speech recognition system, then extracts relevant information to
summarize into a clinical note. The Hybrid scribe condition is
an edited version of the Machine scribe note. The human scribe
condition is written entirely by the medical resident.
To analyze physicians interaction with generate
clinical note, we quantified the number of edits and the scores
that were given. We found that Machine scribe notes received
the lowest average scores. This was followed by Hybrid scribe
notes, and Human scribed notes receiving the highest scores.
We also found that the higher percent in identical note,
meaning fewer changes made, to be correlated with the higher
scores. Subsequently, we also documented the number and types
of changes across the SOAP sections, which you can learn
more about in the paper.
We coded the think-aloud review sessions into several themes to
explain the scores participants gave for each of those
conditions. For Machine generated notes, physicians
found it to be be lacking in accuracy and completeness,
particularly, in the 'Subjective' section. And they
found the content was sometimes placed into irrelevant sections.
For Hybrid generated notes, physicians found improvements in
accuracy, completeness and relevancy but it seemed to lack
medical synthesis, and proper medical terminology. For Human
generated notes, physicians found it to be closest to their
expectations in accuracy, completeness and relevancy. They
were also impressed with the medical syntehsis and second
order thinking that goes into note generation. A minor
irritation found only in Human scribes was that it missed the
personalized aspect, the styles and organizations, participants
have preferred to have in their own notes, for better
comprehensibility.
On the impact of digital scribe systems, physicians all
commented on the efficiency the system can potentially bring. As
p 15 said, "It felt faster. I just had to type in the physical
findings and add a point to the Plan and it was done. That was
just three lines as opposed to an entire page. "
Another aspect physicians noticed was the cognitive load.
Many felt that they needed some form of visual feedback, saying
it would be "easy to forget" what has been already talked
about. But some mentioned, if the notes are generated live in
front of them, they may be tempted to verify what's been
captured and that can be distracting.
When actually using the system and its impact on clinical
workflow, participants were divided into 2 groups. Some
believe that after training they can be more accustomed to
turning the system "on and off", before and after each encounter.
However, others believe that the system should automatically know
when to start and stop since there's already a lot going on
in a clinic.
Major takeaways. Regarding the digital scribe note, first,
accuracy completeness, relevance, especially in the
"Subjective" section are important. When those are
ensured, next, the system needs to work on is including medical
synthesis, then eventually, allow for personalization.
Regarding digital scribe systems as a whole, physicians are
optimistic about its potential, visual feedback is needed but
cannot be distracting, when someone should capture note,
whether that should be automated or retrain into physicians new
workflow is something future systems still need to explore.
This concludes our presentation. Thank you.