Hey everybody, welcome to our CHI presentation on XRStudio: A Virtual Production and Live Streaming
System for Immersive Instructional Experiences. This is work done at the University of Michigan
in my lab, the Michigan Information Interaction Lab and it's co-sponsored by the Extended Reality
Initiative at the University of Michigan. I'm Michael Nebeling, I'm Shwetha Rajaram,
I'm Liwei Wu, I'm Yi Fei Cheng, and we also have Jaylin Herskovitz as authors on this paper. So
a good bunch of authors are present and we wanted to co-present this work to you. So
we started off with some background research to help with the requirements elicitation –
we looked at different taxonomies of MOOC videos and also did a competitive analysis of some live
streaming software to get some ideas there. And then we talked to some key stakeholders to
figure out what they would actually want in a system like XRStudio, particularly instructors
who use XR for teaching, and also media designers who have a lot of experience using
live streaming software and compositing videos of instructors. Our system contains two class
clients one for the instructor and one for the student, and in the instructor client we
object the input of the instructor and then doing segmentation and calibration into one live
composition of the virtual scene, including all the contents from instructors and and
their lecture slides. And then, we store all the live composition into a cloud recording, so the
students can view the contents later or they can join the same session at the same time. In the
student client, they can choose to view the live lecture, replay the stream, or they can replay the
recorded lecture contents in the cloud, and they can also view the lecture content in VR or AR.
Finally, if the students choose join the live stream with the instructor, they can also chat
with their instructor lively, both in text audio and video through WebRTC streaming.
So XRStudio, internally one of the reasons we call it XRStudio is it's actually a
fully self-contained little virtual film studio – you can actually set up multiple cameras.
The default scene that we'll use in some of our examples actually contains a number of different
cameras with angles on the slides, with angles on the 3D content, with angles on the instructor,
both from behind always following the instructor, and then through the slides for the clear board
kind of view. So you can have multiple virtual cameras. There were also ideas that I didn't
fully realize in this paper of actually setting up virtual camera tracks. So really I think in
the future XRStudio could be even a more powerful system to support more professional recordings,
but then you probably also need multiple people. The most important part here, XRStudio can be used
by just an instructor by themselves. Okay, so we ran two evaluations one with students and one with
media designers. So with the media designers, it was with the same group of media designers that
we met with for our formative studies. So we went through a walkthrough with them and elicited some
feedback from that. And with the students, we conducted a remote study with three groups of
four to five masters students, and basically we evaluated three conditions. You can refer to the
paper for more details in regards to the study results we discussed both the advantages and
disadvantages of all three conditions. Generally speaking, students did prefer XRStudio most.
I just wanted to highlight a few interesting findings that I thought were worthwhile
mentioning. So, the mixed reality capture condition was conducive to reducing some of the
feelings of nausea that was commonly associated with first person VR. Slides are still regarded
as a very effective way of delivering information, and many times students wanted that as a reference
as well. And, one kind of limitation we found with the mixed reality capture condition is
students sometimes found the camera viewpoint changes confusing. So that's just to say that the
viewpoint changes must be targeted and this could be remedied with additional feedback beforehand,
to let their students kind of prepare themselves for a viewpoint change. Alright, so this was
XRStudio ,and would have been fun to present it at CHI. It was a fun collaboration involving a
lot of student effort here, and so this was really one of my favorite projects that I've done so far.
So thanks for watching and please reach out if you want to learn more about XRStudio.