Hello CHI23, I am Santawat Thanyadit, and I am happy to present our research, Tutor In-sight:
Guiding and Visualizing Student Attention with Mixed Reality Avatar Presentation Tools.
First, I would like to introduce the motivation of our research.
Many teachers and students use video conferencing technology to work, study, teach, and meet from home. But the video conferencing
systems could only show a live stream from the teacher's webcam.
This made it hard for the teacher to engage with the students
because the teacher could only use the mouse pointer to get the students' attention.
Virtual environments may help solve the issue of low social presence
by providing teachers with interactive meeting places
to engage with their students. Nevertheless, presenting in such an environment
requires a level of technological competence that many teachers may find challenging.
From these observations, it's clear that the current systems don't meet the needs of both teachers and students.
Therefore, we proposed Tutor In-sight, which comprised of a student mixed reality system and a teacher’s dashboard.
In the students' system, we use a mixed reality head-mounted display to show a teacher as a mini 3D avatar
next to the student's printed copy of the materials.
The actions of the Tutor In-sight's Avatar are automatically generated from the teacher's actions in the PowerPoint,
allowing the teachers to continue using their familiar presenting tool with minimal effort.
Tutor In-sight's avatar was designed to use a variety of body language, such as eye contact
gestures, and movement, to direct the students' attention to the presented information such as text, images and 3D objects.
Furthermore, when the student's slide does not match the teacher's presentation,
warning and "blackboard" augmentations are used to help students and display information from the presenting slide.
We also provide teacher’s dashboard to help the teacher gain insight into the entire class's attention.
into the entire class's attention. In the upper-left of the teacher's dashboard,
the teacher can view the students' gaze position overlaid on top of the current PowerPoint presentation.
To assist teachers in better understanding the classroom dynamic, the students are also sorted depending on their status
in the panel and graph below. The teacher may also configure the automated warning message and check specific students
by selecting the student's profile picture. Our study is related to the literature on remote guiding.
Mini-Me, which used mini-avatars to instruct remote collaborators, is the most comparable study to ours.
We also based our design requirement on researches by Woodworth et al.
and Nelbeing et al. There are several works in various areas that we also build upon
so Please see the related work section in our paper.
The key difference between our work and others is that we built Tutor In-sight
so that teachers may use familiar presentation tool, to generate a mini-avatar that can also engage students
From the literature, we derived four design requirements for the Tutor In-sight.
First, we integrate virtual and physical learning spaces
by superimposing the teacher's avatar over the students' printouts,
allowing them to concentrate on a single workspace without splitting their attention
Additionally, if a student looks away from the printout slide when the teacher is pointing at something on the slide,
we have visual cues that direct the student's attention back to the printout slide.
Second, we direct the student’ attention with body language, we utilize mutual gaze,
and avatar movement to help guiding student’s attention to the current teaching subjects. .
Furthermore, we also highlight picture and utilize
3D models to keep the students engage.
Third, we used teacher’s avatar to engage with the students to improve co-presence.
we also utilize textbox to create a sense that the teacher is speaking to the students.
The textbox can be used for both private message and automatic warning message.
Forth, we integrate our system with the PowerPoint and create dashboard for the teacher to monitor the students.
Please see additional detail in our paper
We conducted two user studies, from the perspectives of students
and teachers to determine the advantages of Tutor In-sight.
We invited 6 local teacher participants and 7 remote teacher participants
in the teacher’s perspective study. Most of the teachers are STEM teachers
and familiar with remote presentation.
We recruited 11 local students to join our student’s perspective study.
We asked teacher to compare presentation
tools by giving mock presentations in three different conditions:
videoconferencing, Mozilla hubs, and Tutor In-sight,
During the mock presentation, we ask the teacher to engage with students as if they were teaching the real-class
We used pre-recorded students’ activity to ensure that the experience are comparable across conditions.
From the teacher’s study,
we found 4 keys results. First, while teacher participants felt they could be proficient
VR presenters; six participants believed that the benefits of VR did not
outweigh the effort for daily teaching practice.
On the other hand, nine participants gave positive responses to the Tutor insight
insight condition, stating that the user interfaces were "simple to use",
The second key result is that the teacher
participants think that the eye-tracking are a better alternative to the web camera
in the video conference condition. Six participants
pointed out that it is difficult to supervise students during videoconferences
even when the students switch on the camera, since it is hard to understand what they are looking,
and the teacher felt that they are speaking alone. So, they prefer the eye tracking
since they at least getting feedback that the students are still paying attention
Some teachers also mention the fact that they cannot ask students to open the camera
since some students do not have a private room and opening the camera may violate on their family's privacy.
may violate on their family's privacy.
Another important finding is that students' avatars
and immersion in VR did not always improve teachers' social presence.
As three local teachers noted that the students’ avatars lacked facial expressions
and the avatars alone did not add to the sense of co-presence. Nevertheless, the teacher participants
has two concerns with Tutor Insight dashboard design.
First, student eye tracking data could cause distraction
since there are too many and it moves too quickly.
Second, the eye-tracking data were too specific, which could cause
the students to feel singled out or concerned about their privacy..
Therefore, we redesign the dashboard to help smoothen out the
eye tracking data movements and reduce distractions.
please see our paper for more information
To evaluate the design of Tutor In-sight
from the student’s perspective, we also performed
a within-subjects study comparing the Tutor In-sight condition to the standard videoconference condition
and Mozilla Hubs condition. Similar to the teacher’s study,
we use a pre-recorded lecture to ensure that the experience are comparable across conditions.
From the student's study.
The first key finding is that the students did not enjoy the videoconference condition
seven participants noted the lack of teacher interaction in the videoconference condition.
Another key result is that the Auto-generated avatars of Tutor-Insight and body-tracking avatars (VR) provide comparable co-presence
and attention allocation, since they received similar rating.
Furthermore, seven participants characterized the avatar in the tutor insight condition
as interactive with its body language that kept their attention throughout the presentation.
Three participants also stated that they thought that the teacher was paying attention
to them, which could be attributed to the avatar’s mutual gaze
Another key result is that the note taking ability was hampered in
the VR condition, since they cannot see the real environment.
Nevertheless, the interview revealed different opinions on VR,
Four participants think that VR helped them concentrate, while four other participants
said that the VR environment cause them to feel anxious since their view of the real-world was blocked.
On the other hand, six participants like the integrated
virtual and physical workspace in the tutor insight condition,
which allowed them to concentrate on a single workspace
and improve their note taking ability.
Based on the teacher and students' study, we summarized the top four design requirement
for developing an effective remote presentation system for both teachers and students.
Please check our paper for more information on how we derived these requirements,
And our contribution can be summarized into four points in this slide.
Finally, I would like to thank my co-authors, Matthias and Effie
This publication has been supported by the European Union’s Horizon 2020 research under Project ARETE.
Thank you very much for your attention. I am happy to answer your question now.