Image super-resolution method and system

Abstract

The invention belongs to the technical field of image super-resolution, and discloses an image super-resolution method and system, and the method comprises the steps of processing an image SR through employing a 3D convolutional neural network 3DCNN in combination with the non-local self-similarity of an image, and proposing a 3DCNN-based non-local super-resolution method; modeling non-local similarity by directly utilizing 3DCNN, and extracting non-local similarity information of a natural image; constructing a 3DCNN basic model based on an eight-layer full convolutional network; designing a 3D convolutional neural network in 3DCNN, and proposing an RNN-based improved model, so that a basic model becomes a special example of the improved model. By using the non-local operation provided by the invention, the non-local similar information in the image can be effectively captured, and the SR reconstruction performance is improved; compared with an existing CNN model, the invention shows obvious reconstruction advantages, and is prominent in image scenes with rich structural information.

Description

technical field [0001] The invention belongs to the technical field of image super-resolution, and in particular relates to an image super-resolution method and system. Background technique [0002] At present, learning-based super-resolution has become the mainstream image super-resolution scheme in the past ten years because of its fast calculation speed and excellent performance. It is an ill-posed problem in the direction of computer vision. Different from interpolation and reconstruction methods, data-based super-resolution has richer application scenarios and can obtain higher reconstructed image quality. By learning the implicit mapping relationship between LR images and HR images, and then super-resolution reconstruction of images through this relationship According to the learning object and learning method, learning-based super-resolution reconstruction can be divided into: based on popular learning Methods, learning methods based on over-complete dictionaries, le...

AI Technical Summary

Problems solved by technology

The proposed LapSRN combined the traditional Laplacian pyramid algorithm to design a set of cascaded pyramid deep learning models. The model has the following three characteristics: First, some methods need to use pre-processing before the input image enters the network. A well-defined upsampling operation (such as bicubic) to obtain the spatial size of the object, such an operation adds additional computational overhead, and can also cause visible reconstruction artifacts

Some methods use operations such as sub-pixel convolutional layers or deconvolutional layers to replace predefined upsampling operations. These methods have relatively simple network structures, poor performance, and cannot learn low-resolution images well. complex mapping to high-resolution images

The second is that when using the L2 loss function when training the network, it will inevitably produce blurry predictions, and the recovered high-resolution pictures will often be too smooth

Third, when reconstructing a high-resolution image, if only one upsampling operation is used, it will be more difficult to obtain a large multiple (more than 8 times) of the upsampling factor

[0008] (1) Up-sampling and inputting to the network through interpolation will increase the computational load of the network, and using deconvolution will generate a large amount of computational redundancy, which will amplify the noise while enlarging the image, affecting the quality of the reconstructed image

[0009] (2) The pyramid deep learning model needs to use a pre-defined upsampling operation to obtain the spatial size of the target, which increases additional computational overhead and also leads to visible reconstruction artifacts

[0010] (3) Some methods use operations such as sub-pixel convolutional layers or deconvolutional layers to replace predefined upsampling operations. The network structure of these methods is relatively simple, the performance is poor, and they cannot learn low resolution well. complex mapping from high-resolution images to high-resolution images

[0011] (4) When using the L2 loss function when training the network, it will inevitably produce blurry predictions, and the recovered high-resolution pictures are often too smooth

[0012] (5) When reconstructing a high-resolution image, if only one upsampling operation is used, it will be more difficult to obtain a large multiple of the upsampling factor, and in different applications, it is necessary to train models with different upsampling multiples

[0013] (6) Most CNNI-based SR algorithms do not fully consider the non-local similarity of images, and this property has been proven to effectively improve the reconstruction performance of images in traditional non-local methods

Images