Hi everybody, welcome to our talk!
My name is Konstantin Klamka and today I will present our work “Watch+Strap: Extending
Smartwatches with Interactive StrapDisplays” together with my colleague Tom Horak.
Exactly.
Konstantin and I contributed equally and the research is joint work with Raimund Dachselt
and all three of us are from the Interactive Media Lab Dresden.
Smartwatches are getting more and more popular as always-available companions in everyday life.
However, the input and output capabilities still remain limited.
Especially the display and interaction areas are constrained in their size.
With our work, we explore how smartwatches can be enhanced with interactive StrapDisplays
to introduce new interaction concepts.
Therefore, we investigate how StrapDisplays can enrich smartwatch applications by extending
the interaction space for selecting items in a list or browsing visualizations.
We also looked in several real-world application scenarios like this one with an e-scooter
that uses the StrapDisplays for navigation instructions.
So the main research question for us is: how can we enrich wristwatches while preserving
their familiar and unobtrusive character?
Our approach is Watch+Strap.
As you already seen in the videos our concept focus on adding interactive pixel-based StrapDisplays
to a wrist watch.
If we look at existing work, a lot of research has been done in extending smartwatches with
novel input techniques as well as versatile output capabilities.
But since we aim to visually enhance watches with interactive StrapDisplays, I will briefly
position our work in the context of prior research that investigate on visual screen extensions.
Our first category are transformation approaches that use special folding mechanism.
Watches of this class, like Paddle, WristOrigami, Doppio, or FoldWatch allow to physically enlarge
the display for a temporary time.
In addition, the visual space around a smartwatch could be extended by projections.
While SkinButtons and LumiWatch project user interface components on the forearm, WatchThru
uses and transparent OLED display to extend holographic content above the watch.
MultiFi extends the screen space by using augmented reality glasses.
Another direction of research focus on bracelet screens that are characterized by a smooth
and an enclosing form factor.
Examples are TaskHerder, DisplaySkin or SleeD.
In contrast, on-body display enhancements, like Augmented Forearm, or bracelet assemblies,
like Facet, enable new multi-display arrangements.
The last and most important category for our work are StrapDisplays.
While WatchBand.IO extends the band with LEDs, PrintScreen show how printed electronics could
be used to integrate segment displays onto a watch strap.
Finally, the Sony FES U watch is a commercial fashion device that allows adapt the appearance of the straps.
Unfortunately, the watch lacks interactivity and did not provide any interactive user interface
controls at the watch band.
As we saw, a lot of research has been conducted, however, one promising combination is missing:
A device that combines interactive, pixel-based straps with a watch to provide beneficial
user interfaces and systematically explore strategies for content arrangements and visualizations.
To close this research gap, we introduce Watch+Strap and contribute a design space for combining
interactive pixel-based StrapDisplays with the wrist watch, introduce a comprehensive
conceptual framework and implemented a modular research platform to validate our concepts
based on real-world applications.
So let us start by looking at the design possibilities of enhanced watches
and speak about physical design considerations.
The combination of interactive StrapDisplays with a watch forms a distinct system that
we call Watch+Strap.
This assembly provides a large variety of specific designs.
As a central component it is possible to use a classic watch, a hybrid watch or a smartwatch.
Further both straps can be enhanced with different visual components.
In addition, the physical properties, like the texture of the strap, could play an important role.
For example, it is possible to realize cutouts, use shine-through displays, or integrate a smooth display.
Furthermore, straps could also be enhanced with touch input or other physical controls.
So as we have seen, all three components can feature different degrees of input and output
capabilities as well as different haptic and aesthetic qualities.
In the following, we mainly focus on digital smartwatches and flexible OLED as well as
E-ink screens with touch input.
This selection allows us to discuss and demonstrate several promising interface concepts and arrangements.
Besides the physical design considerations, it is important to consider how to make efficiently
use of a Watch+Strap device ensemble.
Therefore, we systematically looked into the possible display roles, interface types, and interaction repertoire.
As a brief overview, for the display roles we consider how the displays are used and
which of them acts as the main display.
For example, this can depend on the current posture.
For the interface types, we discussed possible components, how they can be arranged across
the ensemble, and how the specific interplay between them can be characterized.
Finally, it is also important to think about how to enable interaction, thus where and
how a user can trigger required functionalities.
In the following, I will illustrate a couple of selected instances of our conceptual framework.
One particular interesting example is continuous content such as a list,
which is used with a focus+context pattern.
This means, the content is spread across multiple displays, and the items that are positioned
on the watch are shown with extended details.
In the video, a user scrolls through a drawer that lists multiple apps,
or here through a music playlist.
For songs in the playlist, additional details such as cover,
album name, track duration, and so on are shown.
Through the focus+context pattern, it is possible to see one item fully detailed, while still
being able to skim over the following items.
Another aspect that was visible in the video, are the simple interaction patterns that we incorporated.
So, when I play the video again, you can see that the rotatable bezel of the watch is used
for scrolling through the list and a tap on either display allows to select an item.
A closely related pattern is overview+detail, which is particularly interesting for content
including visualizations.
In such a pattern, there is not a continuous view but separate ones, where one view is
acting as an overview and one as a detail view.
In the example on the right, a fitness tracker app is used, which shows the recorded metrics
of an activity as line charts.
The overview is placed on the StrapDisplay and allows to identify and select an interesting
part of the data, which is then be shown in detail on the watch.
This pattern has multiple advantages: firstly, it allows to quickly navigate within the data,
while also providing details.
Secondly, overview and detail can be dynamically assigned to the displays depending on which
is better suited for hosting the one or the other,
for example based on the posture or the offered resolution.
In general, each display in a Watch+Strap ensemble has specific characteristics.
For example, the outer strap is pointing away from the user and thus displayed content is
hard to see or requires bending the arm in an uncomfortable way.
At the same time, it is still easily reachable with the other hand.
Therefore, it is suited for hosting quick access controls that can be triggered eyes free.
For example, within a music player such controls can be buttons for the music playback.
By assigning them fixed position on the outer strap, the user can blindly activate them:
the play/pause button is in the middle, and the buttons for next or previous song are
on the left or right.
Further, by offloading the controls, it is possible to use the other displays for showing actual content.
The quick access controls don‘t have to be actual buttons, but could also be triggered through gestures.
For example, swiping up or down on the outer strap can allow for adjusting the volume.
In contrast to the outer strap, the inner strap is pointed towards the user and thus
almost always visible in every posture.
In consequence, this allows to unobtrusively provide information to the user that he or
she can easily glance at.
Such glanceable usages can involve, for example, turn-by-turn instructions when on the go or
notifications when in a meeting.
This is particularly useful when the hands are used for another activity, for example,
when the user has to hold onto a ride or has to carry things.
Such a usage is also interesting with respect to privacy, as the inner strap is easily visible
for the user, but not for anyone else.
These different instances showed the usage types that a Watch+Strap system can enable,
as well as that providing suitable interface mechanisms is essential for such a device.
In order to properly inform our concepts, we followed an iterative development process,
which I will detail next.
This process involved multiple user feedback loops.
After creating initial concepts, we conducted multiple brainstorming sessions, which comprised
unsupervised sketching sessions and supervised brainstorming sessions in groups of three.
The received feedback helped us to refine our initial assumptions.
Across all brainstorming sessions, we had 15 participants in total.
Later on, after we had developed our concepts and realized our prototypes, we invited six
experts for semi-structured interviews in combination with hands-on sessions.
Their comments provided valuable input for further discussions of Watch+Strap devices.
From the six experts, three participated in the earlier brainstorming sessions, while
the other three were external HCI experts.
In general, in all these sessions we could see that such a device is very interesting
to the participants and that they had almost instantly ideas how to use it in specific situations.
This also re-iterates the advantage of having a setup with a familiar and well-liked form
factor that can live up to ergonomic and aesthetic expectations.
However, within this form factor multiple design alternatives could be explored further,
for example, some experts mentioned the idea of incorporating asymmetric straps.
In order to receive such feedback, it was also crucial to develop working prototypes
in the first place.
Our prototypes resulted in an open research platform that allowed us to easily iterate
and evaluate versatile applications.
In order to develop a flexible platform, we decided to use a streaming approach that benefits
from existing web technology frameworks and APIs as well as allows us to separate the
application logic from our different StrapDisplay prototypes.
So technically, we implemented an application server that handles the logic and renders
all necessary views.
All graphics are sent as a bitmap stream to the watch as well as to the StrapDisplays.
Furthermore, all touch and other physical interaction events from the smartwatch or
from the StrapDisplays are immediately transmitted to the server.
All parts are connected to each other via Wi-Fi.
So let us take closer look at the three different types of StrapDisplays that we realized.
Firstly, we realized two StrapDisplays based on ultra-thin, bendable e-ink displays.
Technically, we control the the panels by using an ESP microcontroller board communicating via web sockets.
To physically integrate the displays, we designed
curved watchstrap housings in which the displays seamlessly slide in.
Finally, we sense touch by using three bendable membrane potentiometer
that we mounted under the e-ink display.
As a second type we implemented StrapDisplays with bendable grayscale OLED screens that
are much smaller and actively emit light.
The basic assembly of all components is very similar to our e-ink prototypes.
Finally, we built a prototype to better evaluate the potential of full-color, high-resolution StrapDisplays.
Since the emerging bendable full-color OLED technology is not yet available for prototyping,
we realized an interactive tablet prototype based on a stencil cover
with 3D-printed straps and a smartwatch.
If you are interested in the source code and hardware components of our open research platform,
please check out our project website.
All in all, we believe that the combination of smartwatch and interactive StrapDisplays
forms an interesting device class, for which we provide a design space,
a conceptual framework, as well as a research platform.
With this foundation, we hope to inspire further research of this device class, leading towards more versatile
smartwatches that can then tackle information needs that are currently reserved for smartphones
—all while maintaining the well-known form factor.
In order to get there, we believe future work should explore further design options, such
as asymmetrical straps or additional hardware buttons.
Also technical improvements for miniaturizing hardware components, closing display gaps,
or incorporating fully flexible displays should be pushed forward.
Finally, it is interesting to conduct more extended user studies as well,
for example, to explore the used postures or elicitate interactions.
As Konstantin already showed before, we provide many of our resources online
that can support such explorations.
If you have any questions to our Watch+Strap project, please don’t hesitate to contact us.
And with that, we thank you for watching this talk and are really looking forward to discussing
our research with you in person at some point.