Hello, I'm Florian Jasche from the University of Siegen in Germany
and I'm presenting our study about the comparison of different types of Augmented Reality Visualizations for Instructions
on behalf of my co-authors Sven Hoffmann, Thomas Ludwig and Volker Wulf.
In the past 20 years there were several studies which used AR to support maintenance and manual assembly tasks.
In these studies, the AR instructions were often compared to instructions on paper
or different AR systems like handheld devices or head mounted displays were compared to each other.
The results regarding the key performance indicators like task completion time,
error rate and mental effort are not consistent,
but AR is often seen as a very promising technology for such problems.
However, the actual design of Augmented Reality instructions or more precisely, the holograms
is not very well studied and there is no study which investigates the influence
of different visualization type for Augmented Reality instructions in complex machine set-up tasks.
In our study we are comparing four different Augmented Reality visualization types
for setting up an industrial bending machine using the Microsoft HoloLens.
The first visualization type includes concrete 3D models from CAD Data
to show where and how to mount different parts on the machine
as you can see here on the right side.
The second visualization type only includes abstract elements like arrows and boxes
to indicate where to mount the parts on the machine,
as you can see here on the left side.
In the other two visualization types we use the same holograms,
but we add videos for each step in the instruction.
Furthermore, we used a paper-based instruction as a baseline.
The study was carried out as a between subject study with each 10 participants
for the AR conditions and 8 participants for the paper baseline.
We measured the time for setting up the machine, counted the errors and asked the participants
about their mental effort during the task by filling out the RSME and NASA-TLX questionnaire.
Furthermore, we collected data about the user experience and usability of the Augemented Reality
Applications using the user experience questionnaire and the system usability scale.
We also video recorded the sessions and conducted a closing interview
with the participants for qualitative feedback on the applications.
We performed a statistical analysis of variance on the data with the following results.
The participants with the paper instruction completed the task the fastest
followed by the two Augmented Reality instructions with the abstract visualizations.
Both visualizations with the concrete holograms were the slowest
but there are no significant differences in the task completion time between the different visualization types.
The different visualization types lead to significant different error rates.
Participants with the concrete holograms with additional videos made only 1.7 from 11 possible errors on average.
The abstract holograms with additional videos and the paper instruction performed well, too.
Participants using these visualization types made 2 errors on average.
The most errors occurred with the Augmented Reality visualization with only the abstract holograms with 3.9 errors on average.
This number is significantly higher than for the conditions with videos and the paper-based instruction.
We can also notice a similar effect for the mental effort.
The mental effort was the lowest for the paper-based instruction followed by the two conditions with videos.
The visualization type with only abstract holograms leads to the highest mental effort.
Summarizing our findings
Videos cause no significant longer task completion time
and different visualization types have no impact on the task completion time
The concrete visualization with videos promotes a correct execution and is beneficial for the mental effort
Furthermore, the videos have a high positive effect on abstract visualizations
and they should always be considered when creating Augmented Reality instructions.
The videos have given the participants confidence in their actions.
The AR applications were rated as novel and helpful and no major usability flaws noted by the participants
For the whole results, please read our paper.
Thank you.