Video super-resolution reconstruction method based on multi-memory and mixed loss

Abstract

The invention discloses a video super-resolution reconstruction method based on multi-memory and mixed loss, which comprises two parts of an optical flow network and an image reconstruction network. In the optical flow network, the optical flow between the current frame and the reference frame is calculated for the input multiple frames, and the optical flow is used as motion compensation to compensate the current frame to be similar to the reference frame as much as possible. In the image reconstruction network, the compensated multi-frames are sequentially input into the network, and the network adopts multi-memory residual blocks to extract image features, so that the following input frames can receive the feature map information of the previous frames. Finally, the output low-resolution feature image is amplified by sub-pixel, and added with the image amplified by bi-cubic interpolation to obtain the final high-resolution video frame. A hybrid loss function is used to train the optical flow network and the image reconstruction network simultaneously in the training process. The invention greatly enhances the feature expression ability of the inter-frame information fusion, andcan reconstruct the high-resolution video with rich details.

Description

technical field [0001] The invention belongs to the technical field of digital image processing, and relates to a video super-resolution reconstruction method, in particular to a super-resolution reconstruction method constrained by a multi-memory mixed loss function. Background technique [0002] In recent years, with the emergence of high-definition display devices (such as HDTV) and the emergence of ultra-high-definition video resolution formats such as 4K (3840×2160) and 8K (7680×4320), the reconstruction of high-resolution video from low-resolution video Demand is increasing day by day. Video super-resolution refers to the technology of reconstructing high-resolution video from a given low-resolution video, which is widely used in high-definition television, satellite images, video surveillance and other fields. [0003] Currently, the most widely used super-resolution methods are based on interpolation methods, such as nearest neighbor interpolation, bilinear interpol...

AI Technical Summary

Problems solved by technology

However, the sub-pixel motion compensation layer consumes a lot of video memory, and its effect is very limited

Liu et al. designed a time-adaptive neural network to adaptively learn the optimal scale of time-dependence, but currently only designed a simple three-layer convolutional neural network structure, which limits the performance

Images