Hi!
In this talk, I present the work of me and my colleagues Samantha Straka, Dr. Melanie
Ullrich, Maximilian Ertl, and Prof. Dr. Marc Erich Latoschik.
We show how we analyzed and improved request management in clinical radiology with a rapid
contextual design approach and remote prototype testing.
In clinical contexts, the management of radiology requests is characterized by very complex
and distributed workflows.
These processes often still rely on physical artifacts, such as paper lists, and phone
calls, which frequently end in long call queues.
Relevant patient or case data is difficult to access, which leads to doctors needing
to spend a long time to find the data.
Sometimes, they are not able find the neccessary data at all.
It is time-inefficient and results in frequent interrupts, impacting patient care.
We report on a user-centered design approach of an IT-supported optimization of this workflow.
It is part of an overarching project striving to improve patient care in German hospitals
by using better user interfaces and finding areas that can be improved with AI.
We first aim to rework the underlying processes of radiological request management to be more
user-friendly and to design new, intuitive user interfaces that use this new workflow.
We used the Rapid Contextual Design approach to identify problem areas that would benefit
from restructured workflows and more intuitive user interfaces, analyze user needs, and prototype
solutions.
First, we conducted 21 unstructured interviews with physicians, radiologists, medical assistants,
and student workers.
We then aggregated the data in an Affinity Diagram and conducted a Wall Walk, where we
realized that the interviews did not sufficiently capture the interaction between specific roles.
Thus, we conducted an additional focus group that helped fill these gaps.
From there, we created visions and storyboards detailing an optimized workflow, which we
then analyzed with the stakeholders.
After outlining a new workflow to request radiological procedures, we iteratively developed
interface prototypes supporting this new workflow.
This workflow, however, required us to develop both a mobile and a desktop interface, which
we prototyped as paper prototypes, wireframes, and as a digital version.
Some major problems were long call queues, and patient data being hard to find and distributed
across multiple systems.
We found a mobile app to be most suitable to address most of the found problems.
This approach was especially suitable since every doctor already had a work phone.
It also allowed us to implement functions that would be helpful while on the go, like
managing requests and communicating via chat instead of calling.
This, however, could not be the solution for every role in the process since medical assistants
and students do not have a work phone, and radiologists need high-resolution monitors
for their work.
Thus, we also developed a desktop version of our prototype that supported these roles,
making it possible to plan the procedure from a radiological point of view or analyze the
procedure results.
We tested these interfaces with iterative prototype testing.
We asked participants to think aloud in these tests while working on typical use-cases.
They filled out the System Usability Scale as a marker for usability, the NASA-TLX as
a marker for task load, and some open questions for qualitative feedback.
Due to the COVID pandemic, we switched to remote prototype testing midway through the
testing phase.
We conducted these using Zoom, Miro, and Axure RP.
While we talked to participants via Zoom, we could observe their mouse movements and
how they interacted with the wireframes using Miro.
For the digital version, we used the online sharing capabilities of Axure RP and asked
participants to share their screen.
Overall, 46 participants tested the mobile version, and 44 tested the desktop prototype.
We tested surgeons, physicians, radiologists, medical assistants, and student workers.
Due to scratching the limit of available personnel, we had to test some participants for both
the mobile and desktop prototype.
Both prototypes were well received.
They achieved high usability, with the mobile prototype achieving a score of 87 and the
desktop version achieving a score of 90 while simultaneously causing low task load with
scores of 19 and 13, respectively.
We also asked participants for qualitative feedback about the prototypes.
The participants liked that our prototypes were fast and intuitive to use, the design
was clean and consistent, it had easy access to relevant information, and that the request
process was fast and easy.
Albeit we faced initial problems, remote prototoype testing proved to be useful and yielded equally
good qualitative and quantitative feedback and an even higher participation rate than
in-person testing.
Based on our experiences, we propose five best practices for rapid contextual design
and remote prototype testing in hospitals with complex and distributed workflows:
1.
Supplementing Rapid Contextual Design with focus groups is well suited to identify problems
and solutions in a complex process with multiple roles.
Our initial analysis provided valuable results but lacked an understanding of how
each role interacted with another.
The results of a focus group fixed that gap by showing us these interactions in real-time.
2.
Have a contact person for each relevant role to better understand tasks specific to them.
A contact person in each role did not only provide answers to questions that arose during
analysis but was also helpful to recruit participants and organize rooms.
They were especially beneficial for remote prototype testing since they can fix problems
on-premise.
3.
Be prepared for interrupts and delays in the testing sessions.
Doctors need to be always available for emergencies, which leads to frequent interrupts
and sometimes even delays in testing sessions.
Be prepared for this - saving lives is worth more than achieving high usability scores.
4.
Prepare a workstation with all necessary tools where users can participate in your tests.
We had many problems getting doctors on Zoom calls after switching to remote prototype
testing since hospitals are still low-tech in Germany and doctors lack technical knowledge.
A prepared workstation helped mitigate this by providing all necessary tools and instructions
on using them.
5.
Remote prototype testing works as well as in-person testing.
It is useful if users are hard to reach or scheduling is difficult.
To our surprise, the switch to remote prototype testing increased the number of doctors willing
to participate in our tests because it was easier to schedule tests when they were available,
and doctors paid more attention to their mail accounts.
It also did not negatively affect the testing process or the results.
Overall, Rapid Contextual Design was well suited to analyze and improve the processes
in the hospital setting.
We suspect that our solution might reduce time and resources spent on request management
and lead to increased time for patient care and increased request quality instead.
We plan to compare the current way to request and manage procedures to our solutions in
the future.
Thank you for listening to our talk.
We look forward to your questions and feedback.