An unsupervised video summarization model and its establishment method

Abstract

The invention discloses an unsupervised video summarization model, which relates to the field of video processing in the direction of computer vision. The model includes a conditional feature selection module, a frame-level self-attention module and a conditional generation confrontation network module, wherein the conditional feature selection The module selects the more important frame features in the output video to guide the training of the conditional generation confrontation network module, so that it can pay more attention to the characteristics of this area; the frame-level self-attention module can obtain the long-term dependence of frames in the video , so that the model can better learn global features and avoid removing frames that are visually similar but have a certain time distance; Any manual calibration data can overcome the information loss of the original method, some networks need to be pre-trained, and the model is complex, etc., and can achieve end-to-end training and improve the performance of video summarization.

Description

technical field [0001] The invention relates to the field of video processing in the direction of computer vision, in particular to an unsupervised video summarization model and its establishment method. Background technique [0002] Video summarization is one of the research problems in the field of computer video processing. With the explosive growth of video data in recent years, such as video data captured by massive mobile phones and surveillance cameras, it has brought huge challenges to storing and browsing videos. The purpose of the video summary is to shorten the length of the original video while allowing the shortened video to fully reflect the story line of the original video. The shortened video is also called a video summary (VideoSummary). Video summarization has a wide range of applications, which can reduce the pressure on video storage, save users' time browsing videos, generate highlights of videos, and so on. At present, mainstream methods in this field...

AI Technical Summary

Problems solved by technology

However, the performance of the unsupervised video summarization technology based on reinforcement learning depends on the artificially designed reward function. Although the unsupervised video summarization technology based on the generative confrontation network can learn the confrontation loss function from the data, thereby avoiding the artificial design of the loss function, but The existing methods based on generative confrontation networks have information loss, and some networks need to be pre-trained

[0005] At the same time, the existing supervised and unsupervised video summarization techniques are only based on the recurrent neural network to obtain the relationship between frames in the video, and generally lack the extraction of long-term dependencies of the video

Images