Method and device for quality enhancement of compressed binocular images based on convolutional neural network

Abstract

The present invention is suitable for the image processing technology field, and provides a method for quality enhancement of compressed binocular images based on a convolutional neural network. The method comprises: performing feature extraction of original low-quality images and visual viewpoint images after compression through a first convolutional layer, and obtaining 64 first feature graphs and 64 second feature graphs comprising binocular image high-frequency information; allowing the 64 first feature graphs and the 64 second feature graphs to pass through a second convolutional layer at the same time to allow the binocular image high-frequency information included in the second feature graphs to be merged in the first feature graph, obtaining 32 third feature graphs after merged; performing non-linear mapping of the 32 third feature graphs through a third convolutional layer, and obtaining 16 fourth feature graphs; and performing reconstruction of the 16 fourth feature graphs through a fourth convolutional layer, and obtaining a low-quality image after quality enhancement. The method and device for quality enhancement of the compressed binocular images based on the convolutional neural network can enhance the quality of the low-quality images after reconstruction on the premise of ensuring low transmission and storage cost.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to a method and device for enhancing the quality of compressed binocular images based on a fully convolutional neural network. Background technique [0002] Binocular images imitate the form of human eyes viewing the actual scene, bringing users real 3D visual effects and better stereoscopic immersion, but the transmission and storage costs of binocular images are twice that of monocular images. Therefore, binocular images The target image needs to be compressed before transmission. Referring to the theory of binocular suppression based on the human visual system, it can be concluded that in the binocular vision system, the perceived quality of stereoscopic image quality is mainly determined by the high-quality viewpoint image quality. Therefore, in the case of ensuring a certain stereoscopic image quality, the binocular image can adopt an asymmetric compressio...

AI Technical Summary

Problems solved by technology

[0003] Image compression can be divided into two categories: lossy compression (such as JPEG, Joint Photographic Experts Group, international image compression standard) and lossless compression (such as PNG, Portable Network Graphic Format, portable network graphics), lossy compression will lead to irreversible information Loss, but compared with lossless compression, lossy compression can achieve a higher compression ratio; for example, JPEG uses block-based discrete cosine transform and rough quantization to reduce redundant information between images, thereby achieving high compression ratio; however, image Lossy compression will lead to discontinuous pixel values ​​of adjacent blocks in the image at the edge of the block, resulting in edge artifacts and blurring

For a compressed image, traditional commonly used quality enhancement methods include Adaptive Discrete Cosine Transform (SA-DCT, Shape-Adaptive Discrete Cosine Transform), and Regression Tree Fields-based (RTF, Regression Tree Fields-based), etc., using these methods Reconstruct the compressed image to obtain a higher-quality image, but in this way, some areas of the image will be over-smoothed, resulting in a visual difference between the two viewpoint images

In addition to traditional quality enhancement methods, methods based on deep learning also have good results. Among them, there are methods that use 4-layer convolutional neural networks to learn low-quality (LQ, Low Quality) images and high-quality (HQ, High Quality) images. ) end-to-end mapping between images, and then there is a method to use transfer learning to train a 5-layer quality-enhanced convolutional neural network. In deep learning, the deeper the training network is, the better the experimental results will be; due to the training method or The problem of network design, neither of these two methods can improve the image quality of the network output by deepening the number of network layers; and these two methods are for general image quality enhancement, not for binocular image quality enhancement

[0004] For binocular images, using an asymmetric lossy compression mode can reduce transmission and storage costs and achieve a higher compression ratio, so that binocular images can save code streams during transmission and thus transmit faster, but a single viewpoint The image distortion caused by the high compression ratio of the binocular image will cause visual discomfort to the viewer at the receiving end and the quality of the synthesized middle viewpoint image will be damaged; therefore, after receiving the compressed binocular image at the receiving end, Need to provide an enhancement method to enhance the low quality image at the receiving end

Images