Nonuniform sparse sampling video super resolution method

Abstract

The invention discloses a nonuniform sparse sampling video super resolution method, which belongs to the technical field of video super resolution. The method comprises key technologies such as shot detection, key frame extraction, original image fuzzy processing and down sampling, nonuniform sampling vision model building, sparse dictionary construction, and video reconstruction. A nonuniform sparse sampling method based on foveated vision is adopted, differential sparse sampling is carried out on a video sequence, the number of atoms is greatly reduced, a dual dictionary of high and low resolutions (high and low resolution reference copy) is generated in real time or calculated in a prior mode according to the resolution of a mobile terminal device screen as carried meta data, a hardware-supported nonlinear Mipmap interpolation method is used for simulating and generating a Foveation image in real time, a result similar to that in a Gauss pyramid method can be acquired, and the computation overhead is lower.

Description

technical field [0001] The invention relates to the technical field of video super-resolution, in particular to a non-uniform sparse sampling video super-resolution method. Background technique [0002] Today's mobile devices have become an integral part of our daily lives. Mature wireless communication technology and video codec enable video streaming to run on mobile devices, so people can easily access digital content anytime and anywhere through mobile devices, such as online TV programs, music videos, sports reports and news programs, which Promote the integration of computers, radio and television networks, and various communication systems, make cross-platform video communication gradually become the mainstream method, and initially provide powerful multimedia engines and full high-definition video technology (full high-definition video) support. [0003] However, due to limitations of wireless network bandwidth and terminal processing capabilities, there are still m...

AI Technical Summary

Problems solved by technology

[0010] (2) Current interpolation techniques or upsampling techniques have limitations in terms of reconstruction quality and computation time

Compared with image upsampling, current real-time video upsampling techniques produce more obvious visual artifacts and cannot fully utilize existing computing resources

At the same time, since the target boundary and important details are usually unknown, it will increase the calculation time of upsampling

[0011] (3) Due to the lack of high-frequency prior information and the limitation of computational complexity, the effect of super-resolution technology is unsatisfactory, far lower than the level that theoretical analysis should achieve

In practical applications, this is often limited by the computational complexity, and the individual restoration of each frame of image fails to effectively utilize the useful information of the restored image of the adjacent frame, resulting in a large waste of computation

The method based on iterative and adaptive filtering reduces the complexity of the algorithm by effectively utilizing the effective information of the restored high-resolution image, but this is at the cost of reducing the restoration performance

At present, the problems existing in the super-resolution reconstruction of compressed video mainly focus on three aspects: first, the effectiveness and universal adaptability of the compressed video model need further research; second, the effect of conventional reconstruction methods is not satisfactory, It is far lower than the level that theoretical analysis should achieve; third, the computational complexity of almost all reconstruction algorithms is too high to be implemented in real time

[0012] (4) The traditional wavelet transform cannot effectively capture the smoothness of image edges and contours, so it cannot achieve simplified or sparse image representation

The main difficulties include: how to implement sparse coding in video has certain technical barriers; sparse coding of heterogeneous data in heterogeneous networks has certain difficulties; video data is usually massive data, how to effectively expand the sparse coding algorithm and improve the efficiency of the algorithm , is also a very difficult problem

Images