Robust reconstruction of high resolution grayscale images from a sequence of low-resolution frames (robust gray super-resolution)

Abstract

A method for computing a high resolution gray-tone image from a sequence of low-resolution images uses an L1 norm minimization. In a preferred embodiment, the technique also uses a robust regularization based on a bilateral prior to deal with different data and noise models. This robust super-resolution technique uses the L1 norm both for the regularization and the data fusion terms. Whereas the former is responsible for edge preservation, the latter seeks robustness with respect to motion error, blur, outliers, and other kinds of errors not explicitly modeled in the fused images. This computationally inexpensive method is resilient against errors in motion and blur estimation, resulting in images with sharp edges. The method also reduces the effects of aliasing, noise and compression artifacts. The method's performance is superior to other super-resolution methods and has fast convergence.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims priority from U.S. provisional patent application No. 60 / 637282 filed Dec. 16, 2004, which is incorporated herein by reference. STATEMENT OF GOVERNMENT SPONSORED SUPPORT [0002] This invention was supported in part by the National Science Foundation under grant CCR-9984246 and by the US Air Force under contract F49620-03-01-0387. The U.S. Government may have certain rights in the invention.FIELD OF THE INVENTION [0003] This invention relates generally to high resolution image restoration and reconstruction. More particularly, it relates to a method for computing a high resolution gray-tone image from a sequence of low-resolution images. BACKGROUND OF THE INVENTION [0004] Super-resolution image reconstruction is a kind of digial image processing that increases the resolvable detail in images. The earliest techniques for super-resolution generated a still image of a scene from a collection of similar lower-resolutio...

AI Technical Summary

Benefits of technology

[0007] It is known in the art of super-resolution to introduce a regularization term into the model to help stabilize the solution, remove image artifacts, and improve the rate of convergence. The regularization term compensates for missing measurement information by introducing some general information about the desired super-resolved solution, and is often implemented as a penalty factor in the generalized minimization cost function. A common regularization cost function is the class of Tikhonov cost functions, which is based on the L2 norm and constrains the total image energy or imposes spatial smoothness. This type of regularization term, however, removes sharp edges along with image noise. A regularization term that preseves edges better is the total variation (TV) method which limits the total change in the image as measured by the L1 norm of the magnitude of the gradient. The inventors have discovered that the TV method may be improved by combining it with a bilateral filter to provide a very robust regularization method, which they call bilateral TV. They have shown that bilateral TV not only produces sharp edges and retains point-like details in the super-resolved image but also allows for computationally efficient implementation superior to other regularization methods.

[0009] This computationally inexpensive method is resilient against errors in motion and blur estimation, resulting in images with sharp edges. The method also reduces the effects of aliasing, noise and compression artifacts. The method's performance is superior to other super-resolution methods and has fast convergence.

Problems solved by technology

The inventors have discovered that this least-squares-based approach is not robust, and produces images with visually apparent errors in some cases (e.g., images with non-Gaussian noise).