Hello everyone, today I would like to introduce our work Asteroids: Exploring Swarms of Mini-Telepresence
robots for Physical Skill Demonstration. This work is a joint effort with my wonderful collaborators
Mauricio Sousa, Chu Li, Jessie Liu, Yan Chen, and amazing advisors Ravin Balakrishnan and
Tovi Grossman. Many important problems we face today, such
as environment protection, requires creative solutions from people around the world,
and many of these people live in remote areas with limited resources. An example is Fionn
Ferreira, a high school student who prototyped and built research instruments himself to
remove plastic particles from water. To open up the opportunity for more people like Fionn
to learn the necessary physical skills for their innovations, remote communication technologies
should facilitate the learning of physical skills, such as electronics prototyping.
Remote physical skill instruction today typically involves an expert instructor demonstrating
tools and processes through either live or recorded videos. In this work we focus on
demonstration through live video streams. When livestreaming their demonstrations, instructors
often need to set up one or more fixed cameras. However, as you can see in this recorded livestream,
fixed cameras do not always provide the most informative views.
More importantly, fixed camera setups do not allow audience to actively explore the demonstration
for their individual learning goals. For example, while this camera provides an excellent close-up
shot, it cannot be reconfigured to show the machine used by the instructor, which may
be of interest to some learners.
The other challenge faced by today’s physical skill demonstrators, and instructors in many other domains as well, is lacking awareness
of their audience. What are the students paying attention to? What are interesting and not
interesting to them right now? As Yarmand and colleagues pointed out in a
paper published at last year’s CHI, instructors feel like they are talking into a void.
In the context of physical skill demonstration, instructors may miss the opportunity to adjust
their live tutorials in response to audience’s needs.
Our idea is that mobility and physicality of robotic embodiments can support audience
agency and instructor awareness in physical skill demonstrations.
Specifically, we propose Asteroids, an approach of using a swarm of mini-telepresence robots
at a workbench scale as audience surrogates.
Audience can observe demonstration activities from an robot they inhabit, and find an angle they prefer by adjusting the robot’s position
and orientation.
In the audience interface, users can see the camera live stream of the robot they currently inhabit, a preview for what another robot
sees, a map showing the entire workbench and robot statuses, and direction buttons for
moving the robot they current inhabit.
Audience can freely explore other areas in the demonstration workspace for their individual learning goals.
They can click on the map, and the robot automatically navigates to the target position.
Finally, audience can switch between different robots to get a new perspective immediately.
They can do so simply by clicking on the icon of another robot on the map, and will be automatically transferred to the new robot.
While trying to support audience agency, we also recognize demonstrators’ control of
the tutorial flow, for example, what should be introduced first, what everyone should
pay attention to right now, these can be crucial for effective explanation and instruction.
Therefore, in Asteroids we incorporate a set of tangible interactions for demonstrators’
control of the flow.
Demonstrators can reposition robots simply by grabbing and moving them using the holes on the shell.
They can pick up a red command token and place it on the top of a robot to make
it stationary. Audience can choose to observe through the robot pinned in place, or drive
their own robots. Demonstrators can also place an orange token
on a robot to grab the attention of all audience members. Here, we see an example of a guided
tour through first the 3D printing area, and then the area for assembling individual parts,
and finally, the area for soldering.
Alternatively, demonstrators can place a green token to highlight a region of interest. All
robots will turn to look at the marked region.
Finally, demonstrators can manipulate pairs of zone tokens to define and resize work zones. Robots will automatically avoid work zones
to reduce interference with the current demonstration.
For the implementation of Asteroids, we use commodity webcams and fiducial markers to track robots and tangible tokens. A central server analyzes
the webcam streams and send control commands to robots. Audience connects through a custom
WebRTC application. We 3D-print the shell of the robot, with holes
on the surface for easy grabbing and a dent on the top for placing tokens. We insert an
iPod Touch into the shell as the camera and the network device. Finally, we attach the
shell to a motorized robot base.
We conducted an exploratory online user study using an architectural scale-model building task.
The study involved in total 12 remote audience participants watching an expert demonstrator
showing them how to build an architectural model. The learners attended the demonstration in
a group of 4 through Asteroids robots. The demonstrator largely followed a script, but
with some ad-hoc adjustments for participants’ needs. We collected qualitative and quantitative
data from audience participants.
Participants report that Asteroids recreated an engaging experience of being in a workshop together with the peers and the instructor.
They reported that the demonstrator paid attention to their physical embodiments and this contributed
to their sense of presence. The ability to move around to see different points of interest
also adds to the sense of presence, and help them to obtain a clear understanding
of the demonstrated procedure.
And lastly, participants found the interface easy to use and the features useful.
In summary, we propose Asteroids, an approach of using a swarm of mini-telepresence robots as audience surrogates.
Asteroids leverages robot mobility and physicality to support audience agency and instructor
awareness.
Our interaction design further allows for instructors’ tangible control of demonstration flow.
Our user study with remote audience participants found Asteroids benefitted their engagement, sense of presence, and understanding.
In our future work, we would like to explore the user experience on the
demonstrator's side, and scale Asteroids up for more learners, more robots, and multiple workbenches.
With that I would like to conclude my presentation and welcome any questions. Thank you very much
for your time.