Hi. Our names are Frederik Brudy and David Ledo. And we are very
excited to be presenting our work on SurfaceFleet here. We did this
work during an internship at Microsoft Research, together with our co-authors.
SurfaceFleet is a system and toolkit that explores an interaction paradigm
for migrating content and tools across devices, applications, users, and time.
Given technological advances, knowledge work has become dynamic, spanning a variety of transitions
from one “place” to another. These transitions can be in the form of a physical location ,
migration of work across devices, the presence or absence of a collaborator , or the availability of
a particular piece of information at a given time. To enable mobility of work beyond a single place,
we created SurfaceFleet. SurfaceFleet is a system that addresses these challenges by de-coupling
user interface elements and operations from a particular device, across applications,
for single and multiuser collaboration, as well as at the time that suits best. Our
contribution is part design probe, part reference implementation, and part systems proof-of-concept.
On the UI side, SurfaceFleet utilises Applets, which are semi-transparent
UI elements that float on-top of the existing desktop environment and software.
Because the Applets are always readily available, it becomes possible to work across applications,
synchronously or asynchronously. As a result, SurfaceFleet proposes an
alternative way to think about, design, and implement distributed interactive systems.
SurfaceFleet Applets unbind interactions across four dimensions.
First, Device. Applets run on individual devices,
but their underlying state is preserved through our framework via the Azure cloud. As a result,
applets, as well as their states and properties can be synchronized across multiple devices.
Second, it unbinds interaction from Applications. We enable tool and
content reuse independent of the application. Applets float above the application windows,
and conceptually above the device itself. Therefore, people can operate Applets to
interact with the underlying applications and operating system, regardless of app manufacturer.
Third, User. Since Applets float above the device—and their content can be synchronized—it
is possible to share Applets to further empower individual, as well as collaborative work.
And lastly, with SurfaceFleet we unbind interactions from Time. By
having a robust state preservation, as well as sharing of content and tools,
we consequently support synchronous and asynchronous actions.
And these four dimensions work in tandem, allowing users to selectively share content and tools,
or defer actions until the desired resources are found. It becomes
possible for people to interact with digital content both individual and collaboratively
at the right place and the right time.
So let’s look at some of the techniques supported by SurfaceFleet.
SurfaceFleet
lives in the taskbar. Here you can create and access all available applets. As said, applets
are visual representations of the content ant tools, that float on top of the users’ Desktops
An example of such an applet is a media primitives. Media primitives are pieces
of content, such as images, which can be selected for example from an existing file
or from a camera. Media Primitives can also act as placeholder, to be fulfilled at a later time.
Media Primitives can be dragged directly into individual applications.
Users can operate tools across applications to create a seamless workflow. For example,
a color picker to get a color from reference material
on a different device. These tools can also be
shared to foster collaboration. For example, in a group setting the participants can share
their mouse tele-pointers and keyboards to operate a common large display.
We use “Portfolios”: part artist’s folio, and part portal. Users can
create Portfolios in their own style and it fades into the background when not in use.
Portfolios enable sharing of media primitives simply through drag & drop.
You can also share containers, … as well as tools. A portfolio can reside on multiple devices,
enabling both individual transitions, as well as group collaboration.
For example, you can ask a colleague at a construction site to share their camera
feed through the portfolio. You can watch live, even from multiple colleagues at the same time,
extract images, and use them in your local documents.
Interaction Promises are placeholders for content for fulfillment at a later time.
Interaction Promises offer novel behavior by acting as a reference for future content,
such as an on-site photo that has not yet been taken. Through interaction promises,
it is possible to defer actions or delegate tasks across collaborators.
For example, in Word. Just create a placeholder and pass it along to a colleague. Then,
they can provide the missing information—such as a photo needed from their site visit—whenever
they take the picture on-site. Or you can change your mind later,
and swap the picture for another—the Promise links back and is automatically updated. Similarly,
you can insert a Container Promise, collect images from multiple collaborators, and then select your
favourite to use in your document—or try out different ones until you find the best fit.
Through SurfaceFleet, it becomes possible to break siloed content out of their applications
and beyond the boundaries of a single device. The techniques that we presented—in the form of tools,
inputs, content, and behaviors—offer a variety of distributed user operations,
and enable delegation, deferred actions, and even unstructured workflows.
And we believe, that all this hints at a Society of Technologies that
establishes meaningful relationships amongst the members of this society,
where interactions are unbound from device, application, user, and time.
Thank you so much for attending our talk.