Depth convolution network model of multi-motion streams for video prediction

Abstract

The invention discloses a multi-motion stream depth convolution network model method for video prediction, which comprises the following steps: constructing a new convolution automatic coding networkframework fusing long-term and short-term memory network modules; the motion stream is proposed as the motion conversion from input frame to output frame. The method of generating multiple motion streams at the same time to learn more exquisite motion information is adopted to improve the prediction effect effectively. The base image is proposed as a pixel-level supplement to the motion flow method, which improves the robustness of the model and the overall effect of the prediction. A bilinear interpolation method is used to interact with multiple motion streams on the input frame to obtain multiple motion prediction maps, and then the motion prediction maps are linearly combined with the base image according to the weight matrix to obtain the final prediction results. By adopting the technical proposal of the invention, the time information in the video sequence can be more fully extracted and transmitted, thereby realizing longer-term, clearer and more accurate video prediction.

Description

technical field [0001] The invention belongs to the fields of artificial intelligence technology and video analysis technology, and in particular relates to a motion flow for video prediction and a deep convolutional network model method for video prediction by generating multiple motion flows. Background technique [0002] Video prediction is an important and challenging task in computer vision. Compared with the rise of deep learning in 2012, the video prediction task has a long history. Motion estimation in traditional video codecs already has a prototype of video prediction. However, when the wave of deep learning swept the world, the development of artificial intelligence gave new meaning and new requirements to video prediction. Video prediction in the field of artificial intelligence usually refers to the use of deep learning methods to predict and generate several predicted frames based on the motion information in several frames of video. Generally speaking, video...

AI Technical Summary

Problems solved by technology

However, since such methods generate all pixel values ​​directly based on the features extracted by the convolutional neural network, the prediction effect is directly affected by the feature extraction effect, so the prediction image is very prone to blur

In addition, although this method can theoretically achieve relatively long-term forecasts, it is not easy to achieve clear and accurate long-term forecasts due to the problem of error accumulation

[0005] It can be seen that the existing video prediction methods have the problems of blurred prediction results and insufficient clarity, and it is difficult to achieve long-term prediction

Images