Hello. My name is Richard Li,
and today I will be presenting on AdHocProx.
Sensing mobile ad hoc, collaborative device formations using dual ultra wideband radios.
Before I begin, I'd like to give a quick shout out to my amazing collaborators at Microsoft Research
for hosting me on summer internship and sponsoring this really fun work.
Computing devices play an increasingly important role in our everyday interactions
and collaborations.
However, collaboration across devices remains tedious,
particularly during multi device usage in ad hoc situations
when time is up to connect devices is already limited.
In particular, supporting ad hoc collaboration across mobile devices is challenging.
Requiring users to first configure communication channel between devices,
which can be peer to peer like Bluetooth or a web based cloud service like Google Drive.
And second, once that communication is established requiring users to
navigate a slew of menus to find the desired operation,
such as send this particular photograph.
What if cross device interaction was as easy as physical proximity driving a natural user interface?
We tackled both of these challenges by designing ad hoc proxies,
which uses a dual ultra wide band
or UWB radio system that allows devices to form ad hoc network connections with each
other for peer to peer communication
and enable them to have spatial awareness of neighboring devices,
tracking their position and orientation. For full implementation details,
please see our paper. In the meantime,
several of the highlights include designing a round robin ranging algorithm
for AdHocProx devices to discover each other
and track the position and orientation of remote devices using a machine learning
algorithm on features directed from UWB time of flight measurements.
We found that the UWB signal greatly suffered from interference
caused by human bodies including hand motion,
so including capacitive sensors for detecting grip to mitigate the effects of the interference.
As part of the wireless packets used for ranging,
we also left a few extra bytes for side channel communication.
This allows devices to send data such as user interface updates in a peer to peer fashion.
Example updates include window location and sizing.
With the system in place, I'll now go on to the second part of our work,
the methodology. We designed a study that utilized paper props
attached to our sensors asking people to collaborate
and analyze a dataset printed out on paper maps.
The goals of the study were both to facilitate our
understanding of how collocated collaborators behaved
and to collect realistic sensor data during these behaviors.
As part of the study's protocol, we assign different form factor devices to each participant.
And they had to fold a piece of paper to fit these form factors.
Through this study, we recorded a data set of videos demonstrating various collaborative behaviors,
including individual work, like the top left figure,
working in pairs, as well as full group discussions like the bottom left figure.
All of these exchanges were also prepared with corresponding sensor data.
Our question then became what behaviors do we observe in these sessions that can be supported
and enhanced using the obtained sensor signals.
The first observation we made was that participants
frequently rearranged devices to line up their content.
The constant rearrangement indicated that the
participants had the need to create a larger workspace.
We also observed participants adopt tasks faced the the mission of labor at times,
attributing different roles to their their respective devices.
Such arrangements suggested people needed ways to
flexibly propagate content to other nearby devices.
Our third observation was that devices office serve
as a proxy for the orientation
and relative formation of the user's body relative to their collaborator.
These dynamics offer an opportunity to capture insightful aspects of individual
and group activities around shared physical artifacts.
Finally, During individual work as well as diatic collaboration,
we observed participants orienting their device toward
themselves to signal varying degrees of unavailability.
In contrast, during sharing phases,
participants often place their device in more public orientations to afford viewing by others.
These observations revealed opportunities
for contextual interaction techniques to make collaboration more seamless.
We designed a few interaction techniques that leveraged the advantages posed by our system
and also saw support some of the themes of behaviors we observed in the formative study.
This first technique creates an approval upon physical proximity
and allows users to drag files into and out of it.
In a second study in which participants returned to try the system running in real time,
they reported that this was their favorite technique.
The second technique combines our system with a tilt sensor to enable contextual tilt.
Devices know which remote device, it is being tilted toward.
In this example, a tilt copies a file to the target device.
Finally, the formative study revealed that space was always a commodity,
and this interaction technique allows users to zoom into a
file and have it spin across multiple devices.
Note that these techniques are not the main contribution of its work,
but rather they are examples to showcase
what can be done with sensing spatial awareness and peer to peer communication.
We see our work as a foundation
or platform for others to build and cross device interaction techniques upon.
Indeed, we are excited to investigate how useful some of these design interaction
techniques are in the context of real collaborative tasks.
For instance, we could rerun our formative study with the digital
devices some of these techniques enabled instead.
In this talk, I motivated the need
for computing devices to be able to better support ad hoc collaborative tasks.
I discussed the design and implementation of ad hoc proxy,
our dual UWD system that enables device tracking and communication.
Then I talked about the combined formative study and sensor data collection study we conducted,
observing collaborative behaviors and reporting sensor data.
We took these learnings to guide the prototyping of some example interaction techniques.
We hope that they demonstrated the potential of more seamless
and natural cross device interactions enabled by our system.
Thanks for listening.