in this video we will discuss the paper
publishing kite 20 title from data to
insights a linear storytelling approach
for multimodal learning analytics
analytics was defined in 2011 as an
emerging discipline focus on using
learner's data to create a deeper
understanding about learning with the
purpose of optimizing it nowadays
several learning analytics products have
taken the form of visualizations and
dashboards that had been deployed in a
wide variety of environments however
written reviews have been highlighting
serious limitations in most learning
analytics systems
these include problems that the students
are experiencing in terms of
interpretation since making and acting
upon their data representations in this
paper we focus on an authentic learning
situation in the context of clinical
simulation in which nurses develop and
put into practice critical skills when
looking after a patient mannequins that
need different kinds of clinical support
simulations are often facilitated by a
teacher supervising five or six groups
at the same time our vision is to
automatically capture traces of what's
happening in the simulation classrooms
using sensors to augment teachers
monitoring capabilities or provide
feedback to students in practice these
sensors generate different kinds of
signal for example the physiological
wristbands contain multiple sensors that
generate a true genius data streams of
electrodermal activity blood volume
temperature and wrist movement which can
be used to model constructs such as
engagement or stress similarly x and
y-coordinate generated from localization
sensors coming plotted into a floor plan
but more meaningful data can be obtained
from system block or observation
applications in the form of actions
performed by particular team members a
critical challenge is how to make sense
of all these data and provide meaningful
feedback in ways students and teachers
can actually
Thurston's since they are not at the
experts and most of them will end up
being casual users of learning analytics
interfaces anyway especially in students
who are going to be interacting with an
ecology of all their learning systems
the contribution of this paper is a
layer storytelling approach to provide
guidance to teachers and students one
making sense of multimodal learning
analytics interfaces these model built
from a data modeling technique we
presented a trial last year called
multimodal matrix which relies on
knowledge from domain to map from
multimodal data to a data structure that
has a higher order level of meaning
let's look at an example let me say that
we have in the localization data which
contains x and y coordinates but each
team member in the classroom space and
to give this coordinates contextual
meaning we map each coordinate to
meaningful areas around the patient
manatee
such as being at the best head next to
the patient by the trolley or at the bed
footer in each of these areas
nurses commonly perform specific actions
we transform the positioning data
streams into a discrete data structure
we can do the same for other sensor data
to convert data streams into saline
aspects of communication affective
states and clinical actions information
containing it mainly can be directly
plotted for example as diagrams of stage
of the position of each team member or
the patient in relation to other sources
of evidence but these information is
still not converted in feedback that
nurses can actually make sense of for
improving their own learning for these
we propose to consider the notion of a
learner model and features assessment
criteria to interrogate the multimodal
matrix this is if teachers pedagogical
intention is mapped particular sources
of elements that can be used to assess
how students activity is reflected that
mutation for example
if one teachers intention is that
students effectively perform a cardiac
resuscitation multimodal data could be
interrogated to assess whether students
were the right positions in the right
time and performing compressions
effectively the final component was the
generation of data stories to trade
these at the story
we use a prototype with quality time
with teachers conan-kun timeline
this shows the series of actions
detected by different sensors on a
timeline for each team member in this
case a team of two nurses but is
visually station is inviting students to
explore the various data points
previous studies showed how students
analyze point by point and use their own
criteria to make sense of the
information as a result we call this the
vanilla timeline because it doesn't
offer any guidance to students to
interpreting meaning of the position of
the actions there are time limits their
correctness in combinations of right
wrong actions the visualization would
not update the story we created to data
storytelling prototypes the first word
co.design and then tested with teachers
for simulation in which a patient has
lost consciousness and stopped breathing
we enhance the basic timeline with
ethylene layers for example this one
layer tells a story about the time
responsiveness of two students executing
clinical actions according to the basic
life support protocol we propose to
automatically render visual elements on
top of the vanilla visualization
according to the learning intentions of
the teacher we translated it pedagogical
intention to ahead of rules to add
visual enhancements such as emphasizing
relevant data points while sending
others to the background adding an
explanatory title to communicate the
main message of the story to eLearning
in plain text
faded areas to group data points as
indeed fails to represent slower
responsiveness within in executing chest
compressions to the patient and
annotations
that explain the meaning of particular
data points by highlight in advance
that were important for example a delay
in delivering a resuscitation shock
other layers can be added to highlight
mastitis mail by DT for example if the
airway nurse was not physically located
at the back help during the
resuscitation or showing arousal pigs
automatically detected by the
physiological sensors additionally we
also show the rules used to add
different elements in a particular
letter which were shown to teachers and
students with a point and second
prototype in an authentic classroom
which was shown to the students to
facilitate the reflection after they
perform a simulation that was part of
the regular classes in this case
simulations were about a patient who is
reacting adversely to medication we
focus on the teachers pedagogical
intentions as well for example for this
particular simulation students have to
assess the vital signs of the patient
every 10 minutes we enhance the time
when the device is used to explore all
the data here is an example of a team of
four nurses students had the option to
select the different stories using the
controls at the bottom of the interface
for example regarding the first data
story students check the vital signs of
the patient initially but failed to keep
checking them frequently afterwards
which is explained by the interface
using a combination of visual elements
and narrative it in administer the
medication and immediately realized
there was an advanced reaction and
started but they forgot to perform an
EEG which is recommended the recommended
action after such a situation this error
by omission is highlighted in the
interface and explain to students you
see text we also show text or feedback
about arousal levels automatically
detected by an algorithm based on
students electrodermal activity inviting
students to reflect on for example why
different nurses in this DM experiencing
Cris arousal by the end of the
simulation we analyzed each
students reflections using the lab
protocol which give us a series of a
preset themes of analysis in terms of
the added value of the letters compared
to the vanilla visualization the
envisage classroom uses the effect of
the automated assessment of students
accountability and the impact of opening
the algorithms used by the system most
teachers and students were very
optimistic about the idea of dividing
with potential stories that can be told
basically a time in two layers to
scaffold reflection teachers also
appreciated the potential of augmenting
the kind of evidence that could have him
to provide feedback including aspects
that may be hard to see during the
face-to-face activity three students
also agreed that dividing in data into
sections helped them make sense of the
implications of directions by contrast
the vanilla timer was proven more
difficult to digest students also
emphasized the potential of using the
tool as a complement of the feedback but
they commonly received by the teacher in
the limited class time regarding the
classroom use both teachers and students
struggling highlighted the potential
benefit of using the layer interface to
augment teachers capabilities in leading
evidence-based in class debriefings or
with suitable scaffolding a personal
reflection tasks assigned to students as
homeworld another potential role of the
interface is in the redesign of
simulations
especially if teachers finally record
pain points experienced by students
during the simulations in terms of
accountability it was were a bit worried
about how this information can be
disclosed to students especially because
the students could get them comfortable
in being put on the spot especially if
they made critical errors during the
simulation those strategies were
suggested to address this the
information could be kept strictly
confidential but this would mean other
students will not be able to learn from
others experiences
alternatively the interface could be
identified in share with other students
within the same classroom
to avoid criticism however the students
were surprisingly not uncomfortable at
all and all the students were really
keen about sharing their data stories
for others to learn from their
experiences before they go to real
clinical placements especially if names
are not disclosed all teachers who have
the best students could learn from
opening the algorithm so they could
compare their performance to the
parameters expected according to
national guidelines in terms of
manipulable 'ti of the rules to educator
suggested that those parameters could be
manipulated according to the level of
experience of students but other
educators argue that rules should remain
the same because even first-year
students should be aware of one
suspected from them once they graduate
by contrast the students were less
interested in the algorithm some
suggested benefit of understanding the
expected parameters so they have a frame
of reference but they generally agree
that they're not in the position to
change these parameters
since teachers are the ones who have the
pedagogical expertise
this insight contribute to inform the
growing interest and opening the black
box of the algorithms to teachers and
students to understand to what extent
this can be actually useful from a
pedagogical perspective with summary or
findings demonstrated the potential of
the layer storytelling approach in
helping reduce the complexity of
learning analytics interface instead of
having an interface that divides to
explore the data as analysis analysts do
we can create learning analytics
interfaces that can be used by people
who are not necessarily training data
science and data analysis this has the
potential to lead to deeper reflection
based on objective data especially for
complex multi model analytics of a
ephemeral and physical classroom
activity in the clinical context our
approach has the potential of helping
teachers to revise the instructional
design of the simulations and in
creating context specific analytics that
can
when the feedback that can be provided
for specific clinical scenarios in our
study this was achieved by translating
the pedagogical intentions of the
teacher into visual feedback and
annotations provided reflective guidance
for the specific learning path of hand
evidently or approached and also provide
discussion or individual bias for
interfaces that automatically guide
students or teachers to pay attention to
specific data points the risk of false
negatives is important if certain errors
are not captured students can end up
focusing on mistakes that are not as
critical as others or the wheel and this
is due to be used to support formative
assessment teachers also express both
interest and concern in using the tool
for summative assessment a simulations
are commonly used as safe environments
in which mistakes are allowed and
sometimes encouraged finally ethical
issues related to data ownership sharing
and the identification of multimodal
data well partly cover but this is a
negative deserves further exploration
thank you very much for watching we
might get really fool people to know
more details about our studies and feel
free to contact us thank you
[Music]