This video will present some of the research conducted
in the Graphical User Interfaces for Blind Persons
project funded by the Commission of the European Union.
For users who are blind, the development
of inexpensive machines with command-lined interfaces
allowed them to use computers very successfully.
A one-line command maps easily to synthetic speech
or a braille display for the output medium.
And blind users are proficient with standard keyboards
for input.
Blind users also require some screen-reading software,
which produces the screen text, in either synthetic speech
or braille, and also provides functionality
to explore and review the screen contents, without actually
executing commands.
The increased popularity of GUIs may have made computing easier
for many sighted users.
But for their blind colleagues, this development
has been a drastic step backwards.
Imagine trying to use a GUI if you could not
see its many visual aspects, such as icons, windows,
or the location of cursors on the screen.
Blind users see GUIs as adding an extra layer of complexity
to the interface, which will be of no advantage to them.
Many blind users and sighted experts
believe that the visual metaphors used in GUIs,
such as icons and windows, will be difficult
for them to understand.
Sighted users develop conceptual models of the interface
on the basis of visual information it provides
and their knowledge of objects in the real world.
Blind people effectively manipulate
objects in the real world.
So why shouldn't they use this ability
in the computer interface?
Their conceptual models could develop
on the basis of auditory and tactile information.
One of the great advantages of GUIs for sighted users
is the ability to directly manipulate interface objects.
How can we create interfaces for blind users
in which direct manipulation can be carried out
on the basis of auditory and tactile information?
However, the characteristics of visual information
are very different from those of auditory or tactile
information.
Visual information gives us a very rich,
spatially-organized representation of the scene.
It allows us easy access to both an overview
or specific details.
Like visual information, tactile information
is spatial in representation, giving us
a detailed knowledge of the world at our fingertips,
but is unable to provide the same overview of a scene
obtained from a visual glance.
Auditory information, on the other hand,
has a more temporal organization.
And we can only attend to a limited number of sounds
simultaneously.
These different characteristics of sensory modalities
need to be considered in developing
intuitive and efficient non-visual interfaces
for blind users.
The overall aim of the GUIB project
is to develop interfaces for blind users, which
will give them the same kinds of advantages which sighted users
already have with GUIs, in particular,
to create a sense of engagement, which
comes from the direct manipulation
of interface objects.
To do this, users will need certain kinds of information
about the visual interface, for example, its spatial layout.
Thus, in the GUIB interface, this kind of information
is preserved, rather than presenting
the interface in an alternative form, such as a tree structure.
We are investigating how visual information can be translated
most effectively into a number of different auditory
and tactile forms, in order to cope
with the richness of the original visual display.
We're exploring the use of synthetic and digitized speech,
nonspeech sounds, both natural and abstract musical sounds,
and braille and a touch pad to produce a multimodal display.
In order to give blind users access
to the information in a GUI, the bitmap visual presentation
must be converted into a form with which blind users can
interact.
To do this, the GUIB project has developed a representation
mechanism to describe interaction objects used
in applications and the dynamic changes in these objects.
This so-called off-screen model is
used to generate a text-based presentation,
suitable for non-visual input and output.
Graphical objects are represented
with ASCII characters, such as brackets and text labels.
For example, a checkbox is indicated
with brackets and an x, and the system menu of an application
with an asterisk.
A visual form of the presentation
can also be produced, to enable successful collaboration
between blind and sighted users, for example, at work.
The project has also developed software
to allow blind users to interact with the system
via the off-screen model.
This is equivalent to the screen-reading software
in command-line systems, but must now
cope with both input and output.
Let us now look at how the GUIB system attempts
to create direct manipulation using two
forms of tactile information.
The touch pad gives the blind user
a spatial overview of the screen,
similar to the visual glance of a sighted user.
Touching the pad lightly produces
speech, informing the user of the interaction object
at that location on the screen.
[INAUDIBLE]
Pressing more firmly emulates moving the mouse
to that object--
[BEEP]
[INAUDIBLE]
--which can then be activated with a key,
just below the touch pad.
Direct manipulation at a much finer level of detail
can be achieved using the braille display.
A braille module comprises eight independent pins, in a 2
by 4 matrix, which represents an ASCII character,
by moving the pins up and down.
There is also a small routing button
above each braille module, and pressing it
moves the cursor to this location, a process known
as cursor routing.
The GUIB braille display includes two rows
of 80 braille modules and a vertical display,
based on braille modules, which is
used to give information about cursor positions
and the quantity of information on each line of the screen.
The two rows of braille can display
two lines of the screen.
And, as the cursor is moved in the interface,
by input from the keyboard, touch pad, or routing buttons,
the lines of braille track this cursor.
A fundamental difference between visual exploration
of the display and exploration using braille
is that, whereas the sighted user may
move the mouse about the screen freely,
the blind user can only move the mouse cursor in discreet steps,
by clicking on different routing buttons.
For example, to move the inactive window WinTrain,
the sighted user moves the mouse into the window
and clicks the mouse button once, making the window active.
The blind user, on the other hand,
must click twice on the routing button,
once to move the mouse cursor into the window title
and again to make the window active.
To pick up the window and move it,
the sighted user presses the mouse button
and holds it down, while moving the mouse
to the desired location.
She then releases the button, which drops the window.
The blind user executes the pick up
by double-clicking on any other routing button.
To move the window, she then scrolls down
the display and double-clicks on the destination routing button
to drop it.
These forms of interaction have been
assessed, along with other components of the GUIB
interface, in a systematic evaluation.
This evaluation uses standard sets
of tasks in two common application areas.
16 blind users have each undertaken four hours
of training in GUI concepts and the GUIB interface
and four hours of evaluation.
Extensive objective measures, such as time
taken to complete tasks and error
rates, and subjective measures, such as ratings
of the components, are currently being analyzed.
For example, users were asked to rate the different input
methods.
They rated input from the keyboard most favorably,
probably because they were already familiar
with this method, and it is very reliable.
The touch pad was not rated as highly,
perhaps because users often found
it difficult to locate individual interaction
objects on the small touch pad.
Further design work needs to be undertaken to attempt
to improve its usability.
The routing buttons were very highly rated,
and many positive comments were made about this method
of interaction.
For example, one user said that he
liked using the routing buttons in dialog boxes,
as it gave him a proper hands-on feeling.
A number of other users commented
on how efficient and easy the routing buttons were to use.
These comments are similar to those from sighted users
about direct-manipulation techniques
and indicate that blind users are really
achieving a similar sense of engagement with the system.
Other work in the GUIB project is complementing this research
on direct manipulation.
We are investigating the potential
of a number of novel input devices
and the use of three-dimensional sound and speech output
to create a virtual sound display.
In conclusion, the GUIB project has
shown that there is reason to be optimistic about the potential
of blind people effectively using GUIs.
With appropriate training, particularly
in the metaphors used in GUIs, and suitable interaction
techniques, based on auditory and tactile information,
blind users will be able to have at least some
of the advantages of GUIs currently enjoyed
by sighted users.