Portions of this video are in a stereoscopic format
that requires NTSC video and special glasses.
Without these, you will see a flickering doubled image
during these portions.
This is normal.
Do not adjust your sets.
ARGOS, a Display System for Augmenting Reality,
a joint project of the University of Toronto
and the Defence and Civil Institute of Environmental
Medicine--
"ARGOS" means Augmented Reality through Graphic Overlays
on Stereovideo.
ARGOS uses computer-generated stereoscopic graphics
to enhance stereoscopic video images of the real world.
Instead of trying to create virtual or artificial worlds,
ARGOS creates a reproduction of the real world
and augments it with computer graphics
and can thus display many types of information
in as natural a form as possible.
The main application of ARGOS is in the field of teleoperation.
That is doing a job remotely with a robotic manipulator.
The ARGOS project began in 1987, with the goal
of making it easier to use a teleoperated vehicle.
We identified two key problem areas
with most telerobotic systems.
First, the feedback to the operator
is generally inadequate.
Second, telerobots can be very complicated to control.
We begin with the visual feedback system.
This is the main source of information for the operator.
Most telerobots in use today are equipped
with regular video cameras and television displays.
These systems are monoscopic or two-dimensional,
and working with them is similar to doing something with one eye
shut.
While you can still do the job, it's
a lot harder to see exactly where things are.
Everything appears flat or two-dimensional,
and depth can be ambiguous.
Operators of such telerobots must
learn how to use these 2D images in a way to which most of us
are not accustomed.
With enough training, most operators can do their jobs.
But often, they can't do them quickly or accurately.
Using regular video systems makes
it hard to determine where things are located in space.
The single-eye view means that operators
must watch for shadows and reflections
to figure out where they are.
In this example, the operator can't
tell how far away the can is and in fact is
only able to find the can by hitting it
with the manipulator.
This sort of behavior is not generally recommended,
particularly when working in a hazardous environment,
such as a nuclear power station.
Unfortunately, with regular video systems,
there is no choice.
And now, taking into account the fact
that he misjudged the location of the garbage bin
on the first try, our operator allows a little extra room
this time, only to discover that the claw was
lower than he thought.
In order to overcome the limitations of standard video
displays, we developed a stereoscopic or 3D video
system.
Rather than seeing a flat two-dimensional view,
our operator, when wearing the required glasses,
is able to see a full three-dimensional image.
It is somewhat similar to looking through a window.
Using the stereoscopic system, our operator
can immediately see the relative locations of objects
at the remote site and is able to accomplish his task
successfully on the first try.
Telerobot operators strongly prefer
using a stereoscopic video system.
Many studies have shown that they require less training when
using it and can perform tasks faster and with fewer errors.
Our stereoscopic video system uses a pair
of cameras, one for each eye.
We switch rapidly back and forth between the left and right
cameras and display the alternating images
on the same monitor.
The liquid crystal glasses have electronic shutters that
change from clear to black.
The shutters are synchronized with the cameras
so that through the left lens, you only
see the image from the left camera.
And through the right lens, you only see the image
from the right camera.
The left and right images are slightly different.
Our brains interpret these differences
and allow us to perceive the depth of the image directly.
If there are any glasses available,
please put them on now.
They will fit over most eyeglasses.
The image on the screen is switching back and forth
between mono and stereo so that you can see the difference.
With most monitors, you may notice some flicker.
This is unavoidable.
Standard video monitors are not designed
to be able to produce a flicker-free stereoscopic
image, but they are cheap and readily available.
Flicker-free displays are available,
but they cost considerably more.
It may take a moment for your eyes
to become accustomed to the new image.
Notice how the circular logo seems to actually change shape
and how the tips of the pliers are clearly
curved only in the 3D image.
Our goal is to make telerobots easier to use.
We've looked at the flow of information from the machine
to the user.
And as we've seen, stereoscopic systems
are a great improvement over standard monoscopic displays.
The second part of our work examines
the flow of instructions from the human to the machine.
Currently, operators control the robot manually.
They must devote most of their attention
to driving the machine, generally
using complicated control panels with many switches and levers.
We would like to make the robot more independent
so that we can simply tell it where
we want it to go and let the robot drive itself there.
In order to accomplish this using today's technology,
we need to be able to give the robot precise coordinates.
Operators can't do this directly.
People aren't very good at making
absolute judgments of position.
However, when using a stereoscopic display,
people are quite good at making relative judgments of position.
So if we have a pointer of some kind
and we can detect the current location of that pointer,
we can let the operators move this pointer around
in the remote scene to indicate where
they want the robot to go.
And so using our ARGOS system, we've
developed a computer-generated virtual pointer.
This pointer can look like anything we desire.
Here, we've drawn it as an arrow 20 centimeters across
and 10 centimeters high.
In this version of the virtual pointer,
the computer makes no attempt to interpret the video image
at all.
Instead, it creates a carefully calibrated image of the pointer
so that its dimensions are as realistic as possible
and relies on the human operator to interpret the video image.
When the operator aligns the pointer
with some location in space, the computer
knows where it is drawing the virtual pointer.
And so therefore, it knows exactly where
the operator is pointing.
You will notice that there is a thin line connecting
the current location of the pointer
with the left edge of the chair.
This is our virtual tape measure.
We can use it to measure sizes and distances
in the video scene.
Studies have shown that people are
able to use the virtual pointer with essentially
the same accuracy as a real pointer.
Using our ARGOS system to generate a virtual pointer,
we create a new telerobotic system
in which the computer generates graphics and controls
the robot, and the human operator
interprets the stereoscopic video and gives instructions.
In this way, you make effective use
of the capabilities of both people and machines.
We use human senses to accomplish what technology
can't, and we free the operator from having to control
the telerobot manually.
The virtual pointer can be used in a wide variety of ways.
For example, we can use it to indicate a destination
for an intelligent telerobot.
Or we can draw a path for the robot to follow.
Using a glove input device to control a virtual hand,
we can use computer-generated objects
to manipulate real ones.
The virtual pointer and virtual tape measure
represent the first simple applications of ARGOS.
We can create graphic images of any complexity we desire,
limited only by the speed of our computer.
For example, rather than simple two-dimensional images,
we can generate complex wireframe images
and use these augmented edges to enhance the video image.
In this mockup of a shuttle bay, the graphics
help make the edges within the bay more visible.
Because there is no air in space to scatter the light,
shadows are completely black.
Anything in shadow is invisible.
But if the computer knows what is out there,
either from information stored in a database
or by detecting something with non-visual sensors,
it can augment the video image, enhancing visible objects
and making hidden ones visible.
In a more earthly example, in this scene,
we generate a wireframe outline of a table and chair.
By carefully calibrating our graphics with our video,
we are able to give our operators an accurate
image of where things will be located in space.
If we color our image in, we can make
it appear as realistic as desired,
limited only by the power of our graphics computer.
In this scene, we see a virtual robot in a real environment.
If this were a factory, for example,
it might be possible to teach very expensive
industrial robots their tasks without actually
taking them offline, using a very natural
grab-and-move technique.
This teaching technique could save industry
a great deal of time and money.
This computer desk is being drawn in real time
by a powerful workstation.
If we have even more computing power,
we can make the image indistinguishable
from real objects.
We are just starting to explore the potential of ARGOS.
Virtual pointers, virtual tape measures, image enhancements,
robot control, and object simulations
are just the beginning.
We are currently exploring ways of combining machine
intelligence with human intelligence,
electronic sensors with human senses.
By combining the strongest talents of both humans
and machines, we create an augmented system
in which the whole is certainly greater
than the sum of its parts.
The ARGOS technology you have seen today was developed
in the Department of Industrial Engineering at the University
of Toronto, under contract for the Defence and Civil institute
of Environmental Medicine.
Funding for this video was provided by the Defence
and Civil Institute of Environmental Medicine.
This video is copyright by Her Majesty the Queen in right
of Canada, as represented by the Minister of National Defence,
1993.