A video deshaking method based on deep learning

Abstract

The invention relates to a video deshaking method based on deep learning, which learns a point-by-point map by designing a fully convolutional network, and maps the pixels of the jittering view point to the stable view point point by point according to the map. The method proposed by the present invention is no longer based on the traditional feature matching strategy and homography matrix estimation, but instead performs pixel-level mapping relationship estimation. Such a transformation can solve the problem that the same homography matrix cannot be used locally due to discontinuous depth changes. problem, thus achieving better results in real videos. At the same time, the deep network trained by this method has better robustness, especially when dealing with low-quality videos (such as blurred videos, night videos, and watermarked videos), it has better results than traditional methods. With the help of GPU parallel processing characteristics, the present invention achieves faster processing speed than traditional methods, and can realize online real-time video deshaking.

Description

technical field [0001] The invention relates to a video debounce method based on deep learning, and belongs to the technical fields of computer vision and video debounce. Background technique [0002] In recent years, more and more cameras have been used in various scenes in real life, including a large number of portable and movable camera equipment. The quality of the video recorded by hand-held devices has been greatly reduced due to artificial shaking. The vibration caused by the human sensory discomfort. [0003] Ordinary handheld device recording video deshaking methods are roughly divided into three categories, 2D, 2.5D and 3D methods. 2D methods typically use a sequence of inter-frame matrices to model camera motion followed by smoothing [1]. The 3D method has a better effect on the processing of parallax, by using the motion recovery structure (Structure from Motion, SfM) to estimate the camera path [2], and then using the content-preserving warping (content-prese...

AI Technical Summary

Problems solved by technology

However, its use of a single homography matrix for the transformation from a shaking perspective to a stable perspective limits its ability to solve 3D video

[0004] Over the past few years, CNN has achieved great success in solving traditional computer vision problems, but there are few studies on video deshaking using deep networks.

Images