Hello.
My name is Mark Baldwin, and today I
am going to talk about our paper,
"An Activity-centered Approach to Non-visual Computer
Interaction."
Accessible slides can be found at msbaldwin.com/cscw.pdf.
Tools like electronic Braille displays
and text-to-speech systems, as pictured here,
mediate actions to support goals that a person might not
otherwise be able to complete.
Despite continued improvements to screen readers, screen
magnifiers, as and other assistive technologies,
non-visual computing for blind users
and those with limited vision continues
to lag behind visual computing and usability
in advanced functionality.
One explanation for these challenges
lies in the hierarchical application-centric structure
of graphical interfaces, like the one presented here.
Thus, in this work, we reconsider accessible
technologies for desktop computing from the ground
up as an infrastructure centered around activity.
We applied an activity theory lens
to analyze data collected through a series
of participatory projects working
with and for blind and low vision computer users
and experimental data from a previous study
we conducted on a set of tangible interaction devices,
which are pictured here.
Our analysis focused on the mismatch between activities
and the types of tasks non-visual computing users are
required to perform and the potential benefits
of an activity-centric approach.
So next I'll share some of the results from our analysis.
In a graphical environment, work is often
organized into activity-like structures
through arrangements of applications on the desktop.
For the non-visual user, these activity-like workarounds
are simply not available.
Our analysis revealed that the application-oriented screen
reader inserts itself aggressively
into the workflow of blind and low vision users
making it difficult to reach the levels
of conscious and unconscious operations
that sighted users experience through graphical desktops.
Traditional application document metaphor artifacts,
like documents and collaboration tools as represented
in the diagram here, have no knowledge of each other.
A user conducts actions upon each artifact independently.
And once these applications are closed,
the meaningful connection between the documents
is lost requiring the user to reconstitute the activity when
further actions are required.
Yet, from the user's perspective,
they are core parts of the same activity.
Throughout our fieldwork, we observed orientation
within tracking and pausing of activities
as significant challenges for blind and low vision users.
Finally, our fieldwork showed that the existing translation
of application-centric visual information
was always an obstacle that blind and low vision
users had to overcome.
As applications are opened, they each come
with their own set of commands to operate, configure,
and maintain functionality.
Oftentimes, these commands are unique to the application
creating an additional burden on the user
to memorize functionality at the level of application.
Translation at the level of activity
has the benefit of reducing many of the repetitive commands
that interrupt user interaction as these operations
can occur across all applications rather
than each one individually, reducing
the number of objects in the computing system
that the user must manage.
Next, I'll briefly describe the considerations
designers must engage to merge non-visual computing
with the more natural orientation of human activity.
First, non-visual systems should be redesigned
from the ground up with a consideration
for the fundamental goals of users
rather than act as translators for the task
sighted users might complete to accomplish those same goals.
Second, a consideration for activity
must not end with the initial goal
but must be allowed to adapt and change over time.
And finally, in addition to considering
that blind users might engage in different tasks
than sighted users, an activity-centered approach
to non-visual computing should explicitly
consider the way these different tasks are carried out
across applications.
Activity-based systems and existing research literature
rely on deep integration with the operating system
or customized software to adapt the traditional application
document model to activity.
For non-visual users, this adaptation layer already
exists in the form of tools such as screen reader or screen
magnification software.
Therefore, introducing activity to a non-visual system
is a matter of rethinking how existing assistive
tools present computational information
rather than introduce additional complexity
through new layers of software.
To learn more about the application of these design
considerations, please see our paper
where we present a case study using
an activity-oriented tangible device and software interface
called KIND, pictured here.
Thank you.