Hello, my name is Guoyang.
In this presentation, we will introduce our work in using 3D
projection and Augmented Reality techniques to enhance nursing simulation.
First, we'd like to give a short introduction to nursing simulation.
Nursing simulation is
a widely used teaching strategy for nursing schools to train novices.
It provides an opportunity for students to practice responding to various clinical
situations without putting any risk to actual patients.
It usually involves a well designed
patient care scenario with a simulator and a simulated patient room.
In this slide, we will briefly introduce
each simulator type developed for nursing simulations.
The first type is full body mannequins.
The full body mannequins consist
of multiple programmable components capable of mimicking various human body
functions, such as breathing and eye-blinking.
The second type is part-task trainers,
which only contain a specific body part used to train nursing students
with a specific nursing skill in low fidelity simulations.
The third type is standardized patients
who are actors specifically trained to behave as patients during simulation.
The fourth type is hybrid simulators
that are an integration of the standardized patients and the part-task trainers.
The last type is virtual simulations deployed by either desktops or VR headsets.
The full body mannequin is the most widely used one among all simulators.
However, the full body mannequins do not
have dynamic facial expressions, which limit the mannequins capabilities
in portraying subtitle facial symptoms and interacting with students nonverbally.
To enhance the full body mannequins, we aimed to develop a system that is non
intrusive to students and is low cost to nursing schools.
The full body mannequins equipped with our
system should be able to communicate with students non verbally and portray
facial symptoms that can help students in performing assessment.
To achieve our research and design objectives, we develop a system based
on 3D projection and Augmented Reality techniques.
The proposed system continuously captures a simulation instructor's facial
expressions for generating a corresponding animated face.
The simulation instructor is supposed
to be in a control room and speak for the mannequin through a microphone.
Then the system will project the animated
face onto a 3D printed mask fitted on top of the mannequin's face.
Since some diseased facial symptoms are hard to imitate by healthy human beings
such as facial droop, the system was designed to allow
the simulation instructor to manually manipulate the animated face at runtime.
In summary, our system enables non verbal
communication as it continuously captures and projects the facial expressions
of the simulation instructor who speaks for the mannequin in real time.
In addition,
the system allows the simulation instructor to manipulate the animated face
for displaying any abnormal facial symptoms.
To evaluate the proposed system,
we ran a user study in a stroke simulation scenario with 36 nursing students who did
not have any related simulation experience.
Half of the students were assigned
to the control group without exposure to the proposed system.
The other half were assigned
to the experimental group with exposure to the proposed system,
we aimed to investigate how the system impacted the student's behavior.
simulation performance
and learning experience The stroke simulation scenario requires
students to prepare a patient for knee replacement surgery.
Initially, the students were asked
to finish preparing the patient for the surgery, report to the operating
room, and send the patient for his surgery.
However, five minutes into the scenario, the patient developed initial stroke
symptoms such as giving an alert speech and having facial weakness.
The students should notice the exception,
complete an assessment, and call the provider to initiate treatment.
To investigate how the system impacted the student's behavior in the simulation,
we measured the visual attention for a wearable eye tracker.
Specifically,
since the vital sign monitor, partner, and mannequin are three essential sources
to extract useful cues and make a decision, we investigated how students
distribute their visual attention between the sources.
In addition, to investigate how the system impacted the student simulation
performance, we invited two nursing experts to evaluate the students'
performance using the Lasater clinical judgment rubric.
Moreover, the student must complete a clinical learning environment comparison
survey to illustrate their learning experience during the simulation.
Starting this slide, I will introduce the key findings of our user study.
First, our eye tracking analyses
illustrated that the experimental group students spend significantly more time
fixating on the mannequin and less time fixating on the vital sign
monitor and partner than the control group students.
Then, we will introduce the students' clinical performance.
By comparing the assessment speed and accuracy,
we noticed that the experimental group students performed an assessment and made
a decision faster than the control group students.
However, we also noticed that all experimental group students recognized
the stroke, while four control group students failed to recognize the stroke.
From the nursing experts' point of view,
the face-animated mannequin increased the student's performance in recognizing
deviations but decreased their performance in seeking additional information.
Lastly, we will look at the survey result.
In general, the students reported
that the face animated mannequin was more interactive and helpful for performing
appropriate assessment than the traditional mannequin.
Specifically, 86% of the experimental group students believed that the simulator
they used well met the learning requirements of interacting with patients.
In contrast, only 14% of the control group students had the same thought.
Similarly, 93% of the experimental group students believed that the simulator they
used well met the learning requirements of performing appropriate assessment.
In comparison, only 43% of the control group students had the same thought.
In summary, three conclusions can be drawn from our user study.
First, the students focus more on the novel
mannequin than the traditional mannequin during the simulation.
Second, the system increased the students' performance in recognizing
deviations but decreased their performance in seeking additional information.
Third, the novel mannequin was more interactive and helpful for performing
appropriate assessment than the traditional mannequin.
Thank you for listening to our presentation and this is the end.
If you have any questions related to this
work, please send an email to the following contacts.