- Hi, everyone.
My name is Hiyuan.
I'm a recent graduate from the GEM Lab of
Dalhousie University, Canada.
This work is done with my labmates from Dalhousie
with Thamara and Vinicius from Brazil
and with my supervisors
Kirstie Hawkey and Derk Reilly.
So today I'm very glad to have the opportunity
to share with you the story of how we design
and evaluate tablet interface that enhances
the privacy of content displayed on mobile devices.
So first, let me ask you two questions.
When you read your emails, text messages or browse
your photo albums on your mobile devices in public,
have you ever been concerned about the others
sitting next to you or around you?
On the other hand, have you ever looked over your neighbor's
shoulders and see their information?
So, surveys show that half of the adults in France
are concerned while 72% of UK office commuters
admit to looking over their neighbors' shoulders
on public transportation.
So now that mobile devices are ubiquitous
and they often carry sensitive, personal and work-related
information, it became more problematic
when shoulder surfing happens.
That is someone happens to, nearby, happens
to see your information without your permission.
So our work aimed to provide
digital solution for this problem.
So Brunk has developed a privacy processes framework
for secure, tools and efficiency security and privacy.
So for prevention, we can do this,
or to be more practical, always-on
privacy filters can limit the screen's visibility
to a small range from perpendicular.
However, the need for screen privacy is often transient
and highly context-dependent.
So a more dynamic approach, which detects
shoulder surfing moments, provides awareness
and response on the fly might be more useful.
So awareness could be raised by notifications
in the form of glyph, sound and vibration.
Digital detection has been explored
with front camera-based eye counting and eye detection
or marker-based motion trackers,
electromagnetic motion trackers.
So the way to respond include altering text,
images or general contents.
So our work used to glyph notification,
electromagnetic motion trackers
and general protection techniques.
So the basic idea is we used the motion tracker
to detect the shoulder surfing event.
Make a glyph notification pop-up on screen
and also alter the visual attributes on the content
on your screen to protect the information.
So the questions we asked are,
what notification or protection design
works and what doesn't?
How do we build a digital system based on,
given the user's physical ways to manage their privacy?
And what are, what do the end users respond to
or use our system?
So we studied by designing candid notifications
of different granularity from a simple on/off
to a three discreet intensities at a continuous range.
We also had a literal radar representation
that displays the potential intruder's
relative position and the field of view.
So for protection techniques, we remove color
to grayscale, dim the screen's brightness.
We selectively anonymize sensitive texts or images
and we also leave just a small view port,
but mask the rest of the screen.
So we evaluated this design with 12 participants
in four major privacy scenarios in a busy university library
where the designs are meant to be used.
So we created a seemingly working prototype
with the Wizard of Oz methodology.
So that is, when a participant was working on
his tablet, a wizard used a control interface
to make a notification or protection pop up
to simulate a shoulder surfing event.
So the result showed that finance scenarios
can raise unanimous privacy concerns.
Notifications capture attention
when necessary and not distracting.
So we combined a nondistractive one
and a highly noticeable one
to a new notification and also for a protection techniques,
participants prefer mask to have precise control
in financial tasks and dim were thought to have,
to provide practical protection in all cases.
So we implemented a proof of concept prototype
that monitors an area within the tablet user's vicinity.
We tracked both the on-looker and the tablet
to calculate their spatial relationship.
So when the bad guy is looking towards the tablet
and is within five feet, so the notification
or protection will apply.
So in the real setting, we hung the
magnetic field generator from the ceiling.
We tracked the tablet with a wooden plank
to mitigate magnetic distortion.
Also, we tracked the back of the on-looker's
head with a headband.
Since the data is transmitted wirelessly
and forwarded to a custom application,
which calculates the distance and the relative
orientation in real-time and then communicates
the determined shoulder surfing events
with the user's tablet and apply the notification
and the protection.
So we evaluated this working prototype with,
in the lab with 26 participants recruited in pairs
in four study conditions.
So the baseline is with no digital support,
and the other three are notification only,
protection only and with both of them.
So in the game, so we had three activities.
A game, a personal banking and a collaborative finance one.
So in the game, two participants
played a game against each other.
So the user, tablet user needs to finish a primary task
while try to hide the content, screen content,
and the on-looker tries to see as much as possible.
So in the other more realistic scenarios,
a research confederate played the role of the on-looker
and looked more casually.
So we collected participants' behavior and attitude data
with videos, screen captures, questionnaire and interviews.
So we observed that participants move their body.
They lean over, they angle their tablets.
They look around to gain awareness
and sometimes they tactically utilize
the environment to, as barriers.
So we observed 13 nonverbal and five verbal
privacy behaviors in total.
So using hierarchical cluster analysis,
we categorized the nonverbal behaviors
observed in the game into five categories.
So the top three are response behaviors.
So this behaviors cluster seem to vary in terms of
scale of body movement.
For example, this macro-protection behavior
moves body position and angles significantly.
While microblocking just manipulates the immediate
surrounding of the device that requires less movement.
And device maneuver simply
manipulated the device itself subtly.
So in a more realistic financial task,
we found that privacy behaviors decreased
both in quantity and in magnitude.
Large-scale movement such as macro-protection behaviors,
which was heavily relied on and very effective
in the game, almost disappeared in real-life scenarios.
So our existing physical ways have issues of,
for example, difficulty to stay aware,
unnecessary precautionary behaviors,
less convenience to work at the same time
or advertising user's privacy intentions
or the social concerns of appearing awkward.
So with the support of the digital system,
participants reported significantly higher perceived
awareness, easiness to hide information,
more balance between work
and privacy and overall satisfaction.
So notification was thought to help promote awareness
while protection was seen as an extension,
substitute or alleviators of
socially awkward physical measures.
So participants perceived the management
of digital content through both digital systems
and physical world in a wholistic way.
So first, the privacy management systems themselves
such as physical movements or the changing screen
appearances due to the notification or protection
are also part of privacy.
The second, digital systems were expected
to consider both the user and the on-looker's
physical circumstances.
For example, wholistic system might kick in
when there's loud noise in the environment
or an on-looker is detected beyond the user's vision,
but not when the on-lookers are at an obtuse angle
or their visions are occluded.
Third, so the privacy process are all integral
parts of a whole that physical and digital measures
can be, can collaborate to fulfill people's need.
For example, a person can choose to detect a privacy breach
by themselves while delegating the response to the system.
So half of our participants were from North America
and the other half were from Asia and the Middle East.
So previous researchers argued that
designs that enhance mutual awareness between
the user and the on-looker can exploit social protocols
to regulate both parties' behaviors.
For example, the eye design of the notification
is very obvious and it might make the on-looker back off.
But we found only three participants in North America
mentioned this as a strategy and one North American
participant was concerned about the eye design.
On the contrary, none of the participants
from Asia or the Middle East perceived this mutual awareness
as a strategy, but nine of them used impolite, unfriendly
or not respectful to judge the social appropriateness
of the designs and suggest a more secretive design.
So the trustworthiness of such mobile content
privacy systems became a reoccurring theme in our study.
So 12 participants expressed concern
about the reliability and the accuracy
of the tracking system perhaps due to
the limited tracking range or the magnetic distortion
in the environment.
So five people reported not having confidence
in the dim technique at the beginning,
but changed their opinions later.
So trust is a key factor for perceive the usefulness
and adoption of such systems.
And direct positive experience and more transparency
on the working principles of the technology
can help cultivate trust.
Although we didn't specifically address
how we can practically detect shoulder surfing with our
mobile devices, the feasibility and scalability
of the technology remain to be
the crucial obstacles in this area
So onboard cameras have the problem of limited field of view
or capturing subtle events such as shoulder surfing
with the corner of eyes.
Nonetheless, with improvement of quality of lens,
depths camera or extra sensors such as Google Project Tango,
we believe that such technology will come to reality
sooner than we think.
So we also encourage researchers interested in this area
to join a dialogue and push to the front yourself
the community's knowledge.
So to conclude, we contribute to the privacy theory
by exploring dynamic mobile content privacy management
when prevention strategies are not available.
We provide a justification of physical privacy behaviors.
We also showed that digital systems are useful
when our physical systems are constrained by cognitive
capacity and social norms.
The need for privacy is a delicate matter that
the system designs should be
subtle, social, culturally aware and trustworthy.
Thank you.
(audience applauds)
- [Brian] Hi, Brian Eschell from Kinmount.
Great work.
I liked your, to see your experiments.
- Thank you.
- [Brian] You mentioned the issues
with making this practical
and it seems like one of the main challenges
is reliably detecting the actual user
and distinguishing the actual user from the shoulder surfer.
Have you thought about this at all
how to make this practical or what are your thoughts?
- So, like I said in the feasibility of the technology,
I think, is a major limitation in our study.
But with probably with better quality of lens,
with extra sensorability of our mobile devices,
we might be more able to detect the shoulder surfers.
- [Brian] Alright, thanks.
- Thank you.
- [Taylor] Hi, Taylor Anderson from State Farm.
Question if you gathered any qualitative data from
the participants about whether they themselves
would use this technology after seeing
different versions you showed?
I was thinking about the initial picture you showed
of some folks sitting on the subway, and obviously,
if you had any of those technologies working,
it'll be going off every two minutes, right
'cause you're seeing eyes kind of everywhere.
I wonder if you got any information about
they themselves would actually use this technology.
- Yeah, from both studies we did, we collected
lots of qualitative results and yeah,
the point you mentioned is critical,
especially in busy environments.
So, but participants still thought they would want
this system, but they want the ability
to control the context where they would want it.
So that's exactly because the need for privacy
is highly context-dependent.
Also, we, although we didn't report in the paper,
we had an interview with two physicians from the hospital
and their feedback is very interesting because
the work-related environment is really different
from our personal-related scenarios.
So the work performance would be their first priority
and one person even commented that so if
one person walks by and the system notifies you
and another person walks by, the system notifies you again,
then the doctors would want to
throw the tablet on the floor.
So, yeah, it's definitely very interesting to
evaluate it with people different backgrounds and scenarios.
Thank you.
- [Host] Thank you very much.
(audience applauds)