Method and apparatus for performing a clean background subtraction

Abstract

A background subtraction apparatus of the present invention includes a key point locator for locating key points on a known object type, a boundary point locator for locating boundary points of the known object that make up the edges of the known object, and an edge processor for processing the edges to provide a clean-edged extraction of the known object from a background image. Preferably, the key point locator includes an alignment detector for detecting alignment of an image of the known object type with a skeleton image. Still more preferably, the skeleton image is an exoskeleton image and the known object type is a human being.

Description

CROSS REFERENCE TO RELATED APPLICATIONS[0001]This is a continuation of application Ser. No. 09 / 174,491, now U.S. Pat. No. 6,411,744, filed Oct. 15, 1998 <?insert-start id="INS-S-00001" date="20110329" ?>which claims benefit of 60 / 062,068, filed on Oct. 15, 1997, and claims benefit of 60 / 062,361, filed on Oct. 15, 1997<?insert-end id="INS-S-00001" ?>.BACKGROUND OF THE INVENTION[0002]The present invention relates generally to computer vision systems and digital image processing, and more particularly to the digital extraction of an image from its background.[0003]To produce certain special visual effects, it is often desirable to separate a foreground image from a background image. One method for accomplishing this task is referred to as “blue-screening” and involves placing a blue or other fixed-color screen (e.g. a green screen) behind the desired foreground object, typically the image of a person (the colors blue and green hue that strongly contrasts with most colors o...

AI Technical Summary

Benefits of technology

The present invention is a method for extracting a clean-edged image of an object from its background in an automated and effective manner. It uses an improved method of background subtraction by creating a clean image of the object through improvements in determining its outline. This results in a noise-reduced outline of the object, which can be used for texture mapping or compositing without any holes or gaps within the object image. The invention also includes a key point locator for detecting the alignment of an image of the known object type with a skeleton image, a boundary point locator for locating boundary points of the known object that make up its edges, and an edge processor for processing the edges to provide a clean-edged extraction of the known object from a background image. The key points can include primary key points at the extremities of an image and secondary key points near the torso of an image. The invention has advantages in reducing noise near the edges of an object and improving the accuracy of background subtraction.

Problems solved by technology

One of the disadvantages of using blue-screening for object compositing is that it requires a fixed color screen behind the object.

Another disadvantage is that if any of the colors on the object, such as an item of clothing, are blue, “holes” will appear in the object in the destination image.

This occurs because the pixels in the blue areas on the object will not be labeled as foreground pixels and thus will not be composited with the rest of the object, resulting in the object having holes when composited onto the destination image.

Another disadvantage is, obviously, the need for a special room or screen to provide the appropriate background color.

One disadvantage with this procedure is if a foreground pixel happens to match its corresponding background model pixel color, it will not be considered a foreground pixel.

Another disadvantage is that shadows cast by the object often make the object, when composited, appear to have its original form plus extraneous appendages (as a result of the shadows).

Yet another disadvantage is that if any portion of the background changes or if the camera is moved while the background model is being built, certain portions of the background (e.g. the portions that moved) will be incorrectly labeled as part of the foreground and be composited onto the destination image.

Although there are prior art techniques for updating the background model to reflect changes, they cannot account for a constantly changing background such as one that includes a changing television screen or a window looking out onto a busy street.

As is apparent from the foregoing discussion, it has been difficult to obtain a clean-edged outline for a foreground image being extracted from a background image.

While this is appropriate in the application design stage, it is not appropriate for an end user to have to hand paint or manually create the outline of the image.

Images