This is a prototype of a motion picture authoring system,
which provides a suitable human interface for efficient motion
Picture Editing.
Based on image recognition technology,
the system performs three main functions.
Automatic Scene separation, scene analysis,
and structure editing.
At the first stage called Automatic Scene separation,
separate cuts are identified.
The upper left window shows a motion picture
supplied to the workstation from a VCR or a laser disk player.
By correlating the frames based on their color distribution
as shown in the correlation graph in the upper right
window, a separation of scenes is
performed when large changes in this correlation occur.
Each separated cut as icona fied as a shrunk still
picture and put into the lower window
in the order of time sequence.
Time length of each cut is represented
as variable icon thickness.
The correlation is calculated regionally to remove noise.
Yellow rectangles in the upper left window
indicate the detected regions of large correlation changes.
The second stage called scene analysis allows the user
to keep track of specific camera movements in each cut
through a process called pattern matching.
To analyze a panning or zooming motion,
the picture is divided into 300 subregions represented
as yellow rectangles.
Movement is detected by statistically
comparing the changing position of each rectangle
between frames.
As the green arrow shows, the rectangle regions
shift horizontally when the camera pans.
A radially increasing or decreasing in size
represents a zoom in or zoom out respectively.
This process is highly accurate and automatically compensates
for unintended camera shaking or uneven panning or zooming
speed.
Finally, at this structure editing stage,
the user can freely select any cut
by choosing the appropriate icon from the lower window
and then creating the desired sequence
by arranging the chosen cuts in the upper right window.
Further, the final result can be reviewed and checked
in the upper left window.
Additionally, the user has the option
to play each cut at different speeds to trim it
or to review its detailed descriptions.
The motion picture structure automatically analyzed
and simultaneously visualized in this way
allows the user to intuitively understand the whole structure
at a glance.
Thus, motion Picture Editing is no longer a time
consuming agonizing process.
At the scene analysis stage, the user
can also perform an object extraction from a still picture
when desired.
First, the image is smoothed by an edge preserving filter.
The candidate for the object extraction mask
is obtained by buying ionization.
Next, an image operation is necessary to strengthen
the edge of the object by a derivative operator.
This edge image is subtracted from the candidate mask
to separate it into several parts.
The small black parts, which are portions not required
are deleted by an area size comparison method.
By reversing the operation, the small white parts
are filled in with black.
Thus, the complete mask is obtained
and the object can be cut out from the original image.
This is the result of an object extraction from a motion
picture.
The extraction mask is generated by 2 differential images
of three continuous frames.
The result is used to describe the contents of the cut
or to synthesize an image.