Video prediction method based on spatiotemporal propagation hierarchical codec

Abstract

The invention discloses a video prediction method based on a time-space propagation hierarchical codec. The method of the present invention first samples the given original video to obtain the frame sequence and inputs it to the low-level visual memory encoder to obtain the low-level visual coding features and low-level memory state features; features, and extract high-level semantic features through a high-level semantic encoder; then fuse the obtained low-level visual coding features, low-level memory state features, and high-level semantic coding features through a hierarchical feature decoder to output predicted video frames. The method of the present invention can not only integrate the low-level visual memory features and high-level semantic features, but also make the low-level visual information spread along the time sequence direction through the spatio-temporal propagation module, and can also use the prior knowledge of the first frame of the video to solve the video frame blur problem to a certain extent, Improve overall clarity and visual quality of predicted video frames.

Description

Video prediction method based on spatiotemporal propagation hierarchical codec technical field The invention belongs to the field of computer vision, especially the video prediction technology field in video perception, relate to a kind of A video prediction method based on the spatiotemporal propagation hierarchical codec. Background technique [0002] In the Internet+ era, thousands of videos are generated on various terminal devices every day, and video perception is affected by It has attracted widespread attention from academia and industry, and video prediction is one of the most challenging and highly valuable applications. sense task. This task aims to generate video segments of subsequent moments given a partial video segment. Video Prediction on Radar It is widely used in practical scenarios such as weather map prediction, traffic flow prediction, robot-object interaction prediction, and unmanned driving. example For example, in an unmanned environment, ...

AI Technical Summary

Problems solved by technology

[0004] The shortcomings of the above methods are mainly manifested in the following aspects: First, the method of stacking multiple ConvLSTMs does not integrate the learned low-level visual features and high-level semantic features well, and does not make full use of the learned features, resulting in prediction results There are still ambiguities; second, the video prediction method based on the dual-stream architecture does not effectively propagate the low-level visual features of the video in time sequence, resulting in inaccurate prediction of the position of the object outline in the video; third, the self-recursive method passes the The video frames generated by the network are sent to the network again to realize the video prediction of multiple video frames, but there are errors in the video frames generated by the network, and they will continue to accumulate due to this form, resulting in blurring in the late stage of multi-video frame prediction

Therefore, in order to alleviate the problems of insufficient fusion of different levels of features, inaccurate contour positions of video objects, and blurring in the later stage of prediction, there is an urgent need for a method that can fuse different levels of features, provide more accurate contour position information, and alleviate error accumulation. thereby improving the accuracy of video predictions

Images