Hello everyone, Today I am going to present the project
“Tilt responsive techniques for digital drawing board”.
Drawing boards offer a self-stable work surface that is continuously adjustable
offering artist the freedom to ink, read and draw with several postures.
They could either use it horizontally, or vertically
depending on the task they are performing
During the last decade, new type of digital displays have appeared,
such as the Samsung Space Monitor,
Wacom Cintiq and the Surface Studio that are digital self adjustable drawing board.
Such kind of devices open a new class of techniques that could
sense and respond to tilt adjustments, to allow closer engagement with the content.
Looking at how people use their digital drawing display, a vertical monitor affords reading a
document, but not writing on it with a stylus. Allows reviewing a document, showing it to
colleagues, or to present refined ideas to others.
Instead, angled drafting tables allow close engagement
with the content, allowing to mark-up content, jot down annotation or inking using the stylus.
None of these devices offer a user experience that senses and responds to shifting display angles.
It’s a lost opportunity to appropriately respond to the context of use
But as well to provide different UI based on the context of use.
To address this problem and explore a range of scenarios on the surface studio, we sense display
postures and the continuous transitions in between with a tilt sensor plugged in.
To accomplish that, we streamed data from an accelerometer to sense
tilt motion at the back of the surface. Our application is designed with 3 layers,
the first one that display content, the second one to display ink,
and the last one to capture the input from users.
Through, our app scenarios, we show how a variety of knowledge work scenarios
can use sensed display adjustments to drive context-appropriate transitions.
We prototyped and partially implemented several of our user experiences by playing back
video files interactively, proportional to tilt. The current angle indicates which frame of
the video to display, allowing interactive nonsequential access, forward or backwards,
as the user adjusts the display We found this as a powerful prototyping tool,
because any quick screen recording or video mockup could be used to quickly
try out ideas as you could see where user could annotate a video and a EEG.
We have designed a range of scenarios to summarize the role of display orientation,
input modalities and interaction techniques. Our goals through those scenario is to explore
both continuous and discrete mappings of sensed tilt for self
stable and adjustale digital drawing board. We also wanted to probe how tilt adjustment
could respond to a range of use-cases that people might encounter on a day-to-day basis.
The resulting experiences demonstrate a variety of roles that orientation can play,
as well as how input modalities might change their behavior with display angle.
Our prototype launches to a “lock screen,” displaying notifications,
and from where users can log in into their session. Then as the screen reaches a low
angle, this reveals detailed private information such as notifications.
The Maps application uses tilt to control a perspective view of a city. At vertical,
the screen shows the Skyline, as if looking through the screen onto a miniature cityscape.
As the user tilts down, the view of the city rotates,
until reaching an Overhead View. This experience well demonstrates
our use of the transfer function to transition between low and vertical position.
In another example, we explore the duality between messy/tidy,
and pivate/public while filling sticky notes. While in the low-angled posture, the user can
jot down notes on orange stickies which can be left lying around the screen, like a messy desk.
If a boss or co-worker comes in, you can angle the display vertical
to automatically tidies annotation, and hides personal information.
Our Teleconf App explores a solution that uses the display tilt to transition between
Person Space and Task Space. When vertical, the display emphasizes Person Space. Tilting down
then transitions to Task Space, where the user can mark-up a shared document or whiteboard.
Participants made many positive comments about disabling the camera when tilted or horizontal.
They articulated the benefit of focusing the remote audience on content
The Document App supports fluid transitions between the reading
vs. writing tasks typical of active reading. Users can read and edit text with the display
in vertical orientation. Tilting down then let them handwrite or annotate with ink,
directly on their document, from a convenient display posture.
Participants perceived tilting as less effortful
to automatically call-out comments, and recognize handwritten ink as text,
versus navigating the many UI commands and options typical of current practice in productivity apps.
The Presentation App provides an example where the intended use of the software—authoring or
presenting—can shift with the display orientation. At a low angle, when Authoring, the user can
employ the pen to make private notes or mark corrections directly on the slides
These marks disappear when the screen angles up for presenting,
and the slide expands to fill the display.
A Swipe gestures flip through the slides via direct touch.
The pen also changes roles, acting as a spotlight to call out details
or otherwise direct a colleague’s attention to particular portions of the slide.
The user can point the pen in different directions to cast the spotlight on the
desired area of the screen, proportional to the pen tip’s angle and pressure.
We also implemented a tilt side channel toolbar to solve key issue of surfacing cross-application
functionality associated with tilt, even though tilting has application-specific mappings.
This tilt side channel appear as icon along the side of the screen, suitable for thumb
Side-Channel commands include Clutching, to freeze the current screen,
as well as invoking a general-purpose Annotation Layer.
Considered as an input device, a tilt sensor returns an absolute angle,
in this case between 0 and 90 degrees.. Many of our techniques respond to transitions
in this absolute value, such as moving from low angled to vertical postures.
But for a stable continuous response, small tilt motions must allow for precise control.
In order to stabilize the tilt motion,
we implemented a transfer function that supports both relative, for fine adjustments, and absolute
motion, to maintain a direct mapping between orientation and the semantics of a sensed context.
We divide the motor space of changes in tilt into A, B, and C regions which apply a deadband,
relative control, and absolute control. Within a region A close to this center,
a deadband is applied, so the resulting (virtual) angle used by applications does not change.
As the screen tilts into region B, a proportional gain yields enhanced control.
But if the screen tilts further away into region C,
the virtual tilt angle re-centers by jumping to the absolute tilt angle.
Here is an example of how works how our tilt function works. We can see in blue the
signal smoothed, and in red the raw signal. By dividing the motor space into 3 regions,
we can remove jittering and shaking and allowing for precise control.
We conducted a remote survey of 12 people who currently own a Microsoft Surface Studio,
self-selected via a recruitment message sent to a mailing list of designers.
The survey focused on prevalence and context of screen angle adjustments;
58% of respondents self-reported they adjust their screen daily or more frequently,
17% occasionally, and another 25% only rarely—but with 0% responding “never.”
We then recruited 5 of these participants—selected
to give us a diverse sample based on their survey responses—for in depth interviews.
The purpose of those in depth interviews was to gather impression and feedback.
Participants designated our Document and Teleconf apps as the most compelling experiences. Both of
these scenarios evoke situations where users would already implicitly adjust their screen angle,
typically driven by use of the pen to markup content.
Our Reading vs. Writing transition offers a better writing angle and more space for handwritten
annotations, but also implicitly leverages the tilt adjustment necessary for pen use.
Participants also mentioned difficulties with current telework user interfaces
to find whiteboards and document sharing during video calls.
This suggests that when the UI responds appropriately,
implicit uses of tilt can also lower the perceived effort to access certain program functionality.
These techniques inhabit a continuum between explicit foreground interactions driven by the
user’s intentional tilt motions, versus implicit background sensing techniques that automatically
adapt interface behaviors in situations where the user needs to adjust the display angle anyway.
Hence the most successful techniques strike a balance between the effort require to
adjust display tilt, versus the potential benefits resulting from the transition.
We received mixed responses to our Lock Screen app experiences.
Responses were only positive if the experience dovetailed with a user’s existing practices.
For example, for people who typically put the screen vertical to unlock their screen,
the lock screen did not suit their needs.
The Authoring vs. Presenting transition, and the change in role of the pen, elicited positive
reactions from participants who sometimes present to other people using their Studio
However, for a user who never makes such informal presentations in their office,
automatically switching to presentation mode when tilting vertical would not be useful
A number of our experiences, including the Maps and Live Photos Apps, required explicit tilting.
Here, tilt input acts as a continuous, intentional control.
These experiences contrast sharply with implicit tilting, where screen adjustment serves some
other human-centered task or ergonomic need. It was difficult to remotely assess these
explicit uses of tilt because participants could only watch—not actually engage on these demos
Our work leverages digital drawing boards with 1-dof,
continuously adjustable, self stable displays.
However, other type of display that can be rotated using 3 degrees of freedom should be considered.
Our techniques make the pragmatic choice to sense only the tilt of the display.
But what about the people around it?
To support richer notions of public/private, solo/collaborative, and various nuances in
between, we would need to sense the presence and proxemic relationship of persons nearby.
Taken together, our work contributes an exploration of tilt responsive
application scenarios and interaction techniques for continuously adjustable,
self-stable, pen + touch digital drawing boards.
Our techniques demonstrate both discrete and continuous responses to changes in display angle.
Thanks for your attention.