Time sequence behavior capture box generation method and device based on self-attention network

Abstract

The invention discloses a time sequence behavior capture box generation method and device based on a self-attention network. The method comprises the steps of obtaining a target video; obtaining an input feature matrix according to the target video, and inputting the input feature matrix into a first self-attention model to obtain an action probability sequence, a starting point probability sequence and an ending point probability sequence; wherein the first self-attention model is obtained by connecting a plurality of groups in series, each group comprises at least one self-attention unit, and the target feature matrix corresponding to each group forms feature offset in time sequence; generating a candidate capture box set according to the action probability sequence, the start point probability sequence and the end point probability sequence; and extracting relative position information among the candidate capture boxes, and inputting the relative position information into a second self-attention model to obtain a target capture box set. According to the method, the target capture frame set is obtained by arranging the two self-attention models, and the generation precision of the target capture frame set is improved.

Description

technical field [0001] The present invention relates to the field of machine learning, in particular to a method and device for generating time series behavior capture frames based on a self-attention network. Background technique [0002] Videos in real life often contain a series of continuous complex actions and related events. Based on the research needs of timing behavior in videos, the task of timing behavior capture frame generation has also become a research hotspot. However, how to generate high-quality capture frames for videos and make them The capture frame can cover the real action instances in the video with a high recall rate and a high cross-merge ratio, but it has become a research difficulty. In the prior art, in order to obtain a time-series behavior capture frame, a time-series behavior learning task can be constructed, specifically, a recurrent neural network (recurrent neural network, RNN) and a convolutional neural network (convolution neural network, ...

AI Technical Summary

Problems solved by technology

Among them, the recurrent neural network models the sequence context information through recursive operations. However, the recursive operation is very time-consuming, and the convolutional neural network can be parallelized to achieve acceleration. However, the range of context information captured by stacking multiple convolutional layers is limited. limit

Images