Hello, everyone. My name is Ana Villanueva. I’m a PhD student at the CDesign Lab from Purdue University.
This work was a collaboration with Zhengzhe, Ziyi, and Professors Redick, Peppler, and Ramani.
So what is Meta-AR-App?
It’s an augmented reality-based platform that enables easy authoring for instructors and students.
The main contribution of Meta-AR-App is its unique architecture, which caters to project-based classrooms,
more specifically we implement the pull-based collaboration model,
which allows for real-time authoring and contributions between stakeholders.
So before we move on to the specifics of Meta-AR-App, I want to talk about the background of AR authoring tools.
For a while now, HCI has been trying to make it easier for developers and designers to create rich AR experiences.
Existing platforms such as Unity or Unreal, which are comprehensive game engines,
come with a visual editor and allow assets such as 3D/2D models to be imported and managed.
These are the preferred tools of developers, since they require considerable coding and animation skills.
However, research in recent years has introduced experimental platforms that enable easy-authoring.
Some commercial solutions, such as Vuforia Studio, Blippar, or Layar
have opted for visual interfaces to make the interaction process easy and intuitive;
but these are often limited and work in an isolated space.
So what are these missing?
Well, in one word…Collaboration!
A collaborative architecture that mimics classroom dynamics, which involves sharing information, content, help among peers.
So now, we will briefly go over collaborative tools that are typically used in classrooms.
In this work, we can think of asynchronous editing, such as wikis, or parallel editing, such as google docs.
These tools allow students to read, edit, update content structure between users.
However, these workflows would not work well for an AR classroom because the information would not be time-sensitive
and multiple people could simultaneously modify one step of the project,
which would create conflict and cause confusion among students.
So what could be a good solution?
In open source software, the pull-based model implements collaboration schemes
to streamline the integration of contributions to projects.
The pull-based model became popular within the open source community with platforms such as GitHub.
The typical pull-based model includes: moderators—who are project creators
—that receive contributions from other members of the team
upon pull request and determine whether to merge content based on technical merit or trust.
An adaptation of this model would work well in a classroom setup because it is based on version control,
which was specifically designed to resolve conflict among multiple changes and multiple stakeholders.
Now that we have gone over the background,
I want to talk about our formative study to determine how to cater our platform to the needs of an AR classroom.
Because we wanted to observe the dynamics of a real STEM classroom,
we shadowed a weekly 3-hour session of a project-based class in design and prototyping of smart things,
in which students built their own robots over a semester period.
We actually became very involved with the class material and the instructors.
We also made a scrapbook of our observations from the class
in order to understand how they navigated the content and communicated with each other.
Based on our observation of the class, we concluded with a set of design goals, which we implemented into our system:
these goals included the toolbox that we provided to students to create their own content,
the collaborative architecture that would enable students to help each other,
and a platform that works for the wide range of projects in STEM courses.
So let’s begin by talking about the Instructor Mode of Meta-AR-App:
In the main interface, the functionality of the Animation Palette is similar for our system in Instructor Mode—which is a 2D editor
and Student Mode—which is a 3D editor in the physical world.
However, Instructor Mode provides instructors with more features to moderate the flow, and quality of the information,
this includes taking or uploading video, voice, patterns, pictures, highlighting objects, creating annotations or animations.
Initially, the instructors are in charge of creating the Original Project in the editor,
and then students can make their own contributions as they move along the project.
As such, only instructors are given the capability to accept or reject these contributions for the entire class.
As for the Student Mode, it allows students to start on the application as the recipients.
Once they have locally downloaded the original content, students can modify it as they move along.
In this mode, we only kept essential features: Draw, Annotation, Take Video, and Take Picture.
This was done mainly because uploading file locally doesn’t really make sense during real-time collaboration.
Interaction Modalities of our platform:
In order to alleviate the burden of instructors who serve as the supervisors of the class
and would have to handle pull requests and help out students,
we introduce an original type of pull-based model, "local pull".
Local pull combined with global pull, which is based on the traditional model,
are the two types of interaction modalities facilitated by our platform.
So local pull requests are approved by students in need of help and sent by students who volunteer help.
In this case, students can help out others by adding explanations, such as images, video, and text to the project
and sharing them by pull requests to the cloud.
In turn, the struggling students can browse the suggestions provided by contributors
and choose the most helpful ones to merge while these changes only take effect on their local devices.
Global pull requests are approved by the instructor and sent by the students.
Once the changes are merged, they will take effect globally, meaning that these changes are available to the entire class.
We implement the global pull to help instructors improve the content for a future class.
To clarify, local pull is from student to student; while, global pull is from student to class, pending approval from instructor.
I’m going to talk about the implementation of our system.
Meta-AR-App was developed in Unity. We installed our application in tablets.
We built a cloud server to enable file sharing and communication among users.
We encoded the metadata of AR project files into an XML file
which was uploaded to the sever along with other AR project files in runtime.
The server had another XML file to track the interactions taking place.
Now, I want to move on and talk about our controlled studies for Meta-AR-App.
We recruited 12 participants and split them into two groups: instructors and students.
The instructors were current or former teaching assistants with at least one year of teaching experience in STEM classes.
The 6 students had completed a minimum of one semester of STEM education courses.
The instructors were given scripts—exact procedural paper instructions—of the applications they were supposed to create,
which were taken directly from the first lesson of the smart-prototyping class we worked on and included basic exercises.
The students had to complete the exercises following these instructions.
We gave each instructor and student a tutorial of about 35 minutes on the features of Meta-AR-App.
The participants learned about the main features of our application, including authoring and collaboration.
Then, instructors were in charge of creating an application which included two tasks from this class:
Introducing Basic Electronics and Concepts, and Creating a Voltage Divider.
In both cases, all the features and functionality of our system were tested.
Similarly, the students received the applications created for them by the instructors, and had to perform a series of predefined modifications.
We presented participants with a 5-point Likert scale
1 being strongly agree and 5 being strongly disagree to rate their experience using Meta-AR-App.
As we can observe, overall usability of the application received very favorable marks
and the impressions of the participants were positive as well.
Based on the participants’ suggestions we made some small modifications to our system in terms of simplicity and practicality.
For example, we took out the requirement of instructor’s approval for the local pull,
since this was a time waste for the instructor and not realistic in terms of the pace of the class.
We also wanted to test our system in open-ended user studies.
To this end, we chose 2 instructors with experience in STEM project-based classes.
We gave both a 30-minute tutorial on the features of the system,
the instructors had freedom to design the virtual material for their class using Meta-AR-App.
For this class on IoT development, both chose to teach students how to construct a mini smart-city made out of cardboard material,
conductive ink on plywood for the circuit connections,
and electrical circuitry components (for example, LEDs, batteries, microcontrollers).
The objective of the class was to teach students about concepts
such as polarity, connections in series and parallel, and current flow.
The entire project was comprised of about 25 different actions.
Students received instructions to 17 of them and were expected to explore and figure out the rest.
The instructors designed all the material for the class, including the cardboard pieces for the smart-city, circuitry logic, and the procedural AR.
We recruited 40 undergraduate students. And we split them up into four sessions:
No-AR: a typical class with the instructors, in which the main instructor taught in front of the class,
while showing the instructions on assembling the smart-city by using a projector-view of his hands.
AR-only: a class with AR content created with Meta-AR-App
made up of the procedural assembly of major components of the smart-city.
AR-local: a class with AR content created by the instructors and contributions,
such as help, hints, answers, suggestions from student to student (this being local pull) using our platform.
And finally AR-local-global: instructors selected contributions from students (this referring to global pull)
to improve the AR content created by them,and the option of continuing contributions from student to student (this being local pull).
We can observe that introducing AR into the classroom decreases overall error per class during problem-solving.
Also, there was a large number of contributions for the AR- local and the AR-local-global conditions
from students to their peers which were requested and answered or volunteered.
Similarly, number of help requests for the AR conditions mimics the amount of interaction in a non-AR classroom.
So our discussion centers around what a classroom which implements Meta-AR-App looks like.
We organized our class model into four categories based on our user studies: Work, Design, Collaboration, and Technology.
Work: Manipulating tangibles
AR is actually really helpful to visualize abstract concepts, and in our case students took advantage of that
and student in the local and global conditions made sure to share anything they felt wasn’t shown like current or polarity.
So these became visible rather than just trial and error procedures.
Design: Creating Learning Content
The implementation of the pull-based model gave instructors and students
the valuable tool of creating AR content in an AR environment in real-time,
obviously pending approval from a moderator. But this made the quality of the class better.
And then Collaboration: Between Students and the Class
Obviously students’ first instinct was to ask questions to the instructors regardless of the collaborative technology being implemented.
Once collaboration was implemented for the AR- local and AR-local-global conditions,
we observed that students providing contributions, such as help or hints,
were typically the most advanced in the assignment and they made up about 30% of the class.
This is different from collaboration conditions based solely on proximity, in that the software allowed
for the best students to actively engage in helping the struggling students that were not sitting close to them.
This type of collaboration aided the instructors: by relieving them of the pressure to help students one at a time
and by directly providing help to the struggling students that was accurate and timely.
An observation from the class was that the number of contributions was higher for both AR local and AR local-global conditions
as compared to the number of help requests from students.
So this meant that students were actually making voluntary contributions rather than just answering questions.
Technology: Effective Learning and Problem Solving
Once the local and local-global conditions were implemented, the debugging process became a collective experience,
in which students were contributing with possible ideas on how to solve the circuit.
Finally, the AR-local-global condition was the most student-friendly condition
—this was the second iteration of the original AR project created by the instructor
--mainly because the AR content considerably improved based on contributions made by the students from the AR-local session.
To finalize, I want to end with these key points:
Meta-AR-App is an authoring platform for collaborative AR.
We demonstrated how we can leverage the medium of AR
combined with cloud technologies to support selective and timely collaboration,
which enables a decrease in error during problem-solving.
We also observed how iterative improvement of the AR learning content in the form of global pull and local pull,
can improve the original AR project and spark curiosity and creativity among students’ learning process.
Thank you for listening! If you have any questions don't hesitate to let me know.