Hi everyone, today I am going to talk about AdapTutAR: An Adaptive Tutoring System for
Machine Tasks in Augmented Reality.
Modern industry shows an increasing demand for flexible and self-configuring production.
This poses challenges for training workers to rapidly master new machine operations and
workflows, or simply called, machine tasks.
Augmented Reality has been widely adopted in machine task tutoring systems.
Some tutoring elements have been proven effective and welcomed by end users, such as text, animated
components, and avatars. Although these AR tutoring systems have the
benefit of spatial and contextual visualization, they are not adaptive to a learner’s real
time behavior and prior knowledge. As a result, the learning is less effective than using
a human tutor.
To improve the learning outcome, we turn to the area of intelligent tutoring in which
many approaches have been proposed to adapt the tutoring content.
In general, researchers have considered diverse sources of adaptation, such as learner’s
performance and prior knowledge. Also, researchers considered various targets
of adaptation. In this paper, we identified several sources
of adaptation that are suitable for machine task tutoring in AR.
Accordingly, we choose four tutoring elements.
Specifically, we have avatars, animated components and arrows, expectations, and subtask descriptions.
To better organize these elements to deliver an efficient learning experience, we propose
AdapTutAR, an adaptive AR tutorial system for machine tasks.
We demonstrate the entire process using a laser cutting machine task.
In the first step, an expert creates a tutorial in AR through embodied demonstration.
Next, the expert can preview the avatar and animated components, and choose to edit the
tutorial, such as adding expectation or subtask description.
Now we have four tutoring elements. When learner starts to follow the tutorial,
AdapTutAR adjusts the tutoring elements in terms of five possible levels of details.
When the level is 5, the system shows all four elements. Next, hide avatar and show
the other three. Next, only show expectations and subtask description. Next, only show subtask
description. And finally, no elements.
We built an adaptation model based on deep learning and finite state machine to adjust
the level of detail.
First, the learner should open the machine lid.
The current Level of Detail is 3, the system detects that the learner is stuck and increases
the Level of Detail to 4 so that an additional animated component appears.
Now, the system detects a correct operation and adjusts the Level of Detail to 3 for this
step and all other similar operations. The next operation is picking up the cardboard.
The system recognizes that the learner is not paying attention to the target, and increases
the Level of Detail from 4 to 5 to bring out the AR avatar.
Following the avatar, the learner finds the cardboard and picks it up.
As the learner repeats the tutorial, AdapTutAR gradually decreases the Level of Detail according
to their performance. At level 2 and 1, no step-by-step hint is
provided, so the learner is encouraged to recall and perform that task by themselves.
Here, when they forget to adjust the laser head and try to close the lid directly, the
system increases the Level of Detail to inform them.
In the end, with the help of AdapTutAR, the learner masters the task without any tutorial.
To evaluate the performance of the adaptation model, we conducted a 2-session remote user
study in VR. 24 users learned 2 machine tasks using AdapTutAR
and a non-adaptive system as the control group. The non-adaptive tutorial shows all four tutoring
elements to the users all the time. The users first learned the tasks following
the tutorials, and then, did a test without the help of the tutorials.
The results show the adaptive tutorial leads to a better learning outcome in terms of completion
time and mistakes. Please refer to the paper for more details.
Thank you!