[MUSIC PLAYING]
CoMedi, using computer vision to support awareness and privacy
in media spaces.
Media spaces such as cave cat, cruise, montage, or the NYNEX
porthole are intended to facilitate
informal communication and group awareness while assuring
privacy protection.
However, low bandwidth communication
is a source of discontinuities in the communication process,
resulting in a loss of peripheral awareness.
Privacy as in rave is often implemented
as an accessibility matrix coupled
to an all or nothing exposure of personal state.
In addition, current technical solutions
have no provision for configuring
the user interface in relation to the availability
of resources.
CoMedi is a media space prototype
that addresses the problems of visual discontinuity, privacy,
and resource configuration using a combination of computer
vision and speech.
Before showing these three original contributions
of CoMedi, we first present the overall design solution.
The driving design requirement for CoMedi
has been supporting awareness.
A fisheye porthole and an events log
are the primary means of addressing awareness.
Let's see them in action.
By glancing at the screen without any extra action,
I can observe that six persons are currently
present in my porthole, that all of them are available,
the dot [INAUDIBLE] is green, except Joelle
in the bottom left corner whose dot switches from red
do not disturb color to a yellow busy color,
and that meanwhile, someone has entered the Francois' office
in the bottom right slot.
At the bottom of the screen, the event log
tells me that some people have recently joined the media space
using the machines comet, hamlet, and eprouvet,
whereas William has just left.
I am intrigued by Francois' office.
To discover more, I can enlarge the slot using
the fisheye control of the porthole,
switch to another office, enlarge the view,
or return to the normal rendering mode.
Having presented the general picture of CoMedi,
let's see how we address visual discontinuity.
Our first technique to alleviate visual discontinuity
is the use of a multi resolution fovea.
This technique is inspired by the architecture
of the human retina.
The image presented to the user is a composition
of a high resolution fovea and a low resolution periphery.
The fovea is provided by a high resolution steerable camera
while the periphery is given by a low resolution fixed camera.
Using the mouse, the user can place visual attention
on a region of interest without loss of context.
The fovea can be zoomed to provide
the required level of detail.
As shown here, the fovea may be circular or rectangular.
The composition of a fovea and the periphery
avoids the visual discontinuity that
results from a split screen, such as in the system
extra eyes of Buxton.
In a user study, six users were asked to find random targets.
All of them preferred the circular fovea.
However in all cases, a rectangular fovea
was more efficient than either a circular fovea or extra eyes.
Visual discontinuity may also be alleviated by face tracking.
Let us see it in action in a V phone.
Eric has selected Joelle's slot and calls the V phone function.
Now that Joelle is on line, Eric stands up
to show her something on the wall.
The camera follows him as he moves.
The next sequence shows how the tracker works.
The robustness of the face tracker
is based on a new approach that uses a supervisor to coordinate
multiple tracking techniques.
Here we use eyeblink detection, correlation,
and skin color detection.
This next sequence shows skin color detection in action.
The left image shows live video of Eric.
The green rectangle expresses the location of Eric's head.
In the right image, screen denotes
the pixels that have a high probability to be skin.
The implementation shown here runs
at more than 20 images per second on a Silicon Graphics
Indy.
The robustness and speed of the tracking system
support fluent interaction.
As in many systems, privacy is supported
using an accessibility matrix for specifying access rights.
Publication and filtering of private data
offer complimentary and original ways to provide privacy.
Strong protection is provided by hiding using the publication
mechanism.
For example, Eric removes Joelle and Frederic
from the list of people to whom he publishes his private data.
On Joelle's porthole, Eric's slot turns to the private mode.
His level of availability and video
seen are no longer observable.
Instead of hiding, users can choose
a less radical protection such as setting filters
on their video seen.
Here Eric has set up the activity filter
based on a difference of images.
The filter passes motion and thus supports both privacy
and group awareness.
Using the ViaVoice, voice recognition system,
he now selects the Venetian blinds,
then removes the filters.
[NON-ENGLISH SPEECH]
He can also use an innovative new filter
called an Eigen space filter.
The top left image shows live video of Francois.
The top right image shows the filtered image
received by remote users.
A filter is defined by an orthogonal basis
space of images computed by principal components
analysis on a set of socially correct images
provided by the user.
As shown here, events not included
in the socially correct image set
are removed by the projection.
Users want to know, who has glanced at me?
Experience with the NYNEX porthole
indicates the necessity for users
to ask the system who has glanced at them.
As shown here, CoMedi provides this function in a simple way.
Provision for reconfiguration of resources
is our final contribution to media space technology.
Firstly, interaction techniques can be configured depending
on computing resources.
For example, at the beginning of a session,
I can select or eliminate the speech recognition
system as well as the computer vision tracker.
Because a media space runs permanently
and because the screen is a critical resource,
CoMedi provides ways to manage screen space.
Users can build their own porthole
keeping the most important people
in the mosaic while saving computing resources and screen
real estate.
For example, here I have five people in my mosaic.
I can reduce the bandwidth and the computing load
by removing Stephane.
[MUSIC PLAYING]
In summary, CoMedi is a media space prototype
that integrates computer vision and speech recognition.
CoMedi demonstrated solutions for the problems
of visual discontinuity, using a multi resolution
fovea and a face tracker, privacy,
using filters such as the Eigen space filter,
and user selected resource configuration.