Group behavior recognition method based on graph neural network

Abstract

The invention discloses a group behavior recognition method based on a graph neural network. The method comprises the following steps: feature extraction: extracting the individual visual features ofa video segment in unit time, and obtaining the feature expression of each person and the feature expression of a whole scene; generating a virtual graph, generating a fully-connected undirected graphaccording to the obtained feature expression and scene feature expression of each person, and introducing virtual nodes into the undirected graph to generate a virtual graph; graph neural network updating: performing graph neural network updating on the virtual graph; constructing a graph neural network, and constructing a graph neural network model according to the graph neural network layer; and group behavior recognition: importing the complete virtual graph into a graph neural network, and carrying out error calculation on the prediction class label and the real class label. According tothe method, a novel graph neural network based on virtual nodes is defined, rich time-space features in the video can be learned, and therefore group behaviors in the video can be accurately recognized.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular relates to a group behavior recognition method based on a graph neural network. Background technique [0002] Given a video, the intelligent recognition system needs to recognize the group behavior involved in the video. This involves analyzing the content of the video both spatially and temporally. [0003] At present, the main recognition method is to input video clips into a three-dimensional convolutional neural network, and the three-dimensional convolutional neural network performs feature extraction in three-dimensional space, and directly outputs the recognition results of the video. Or extract multiple frames of pictures in the video, input a two-dimensional convolutional neural network for each frame of pictures, and use the two-dimensional convolutional neural network to judge the recognition results of each frame of pictures, and average the recognition results of mul...

AI Technical Summary

Problems solved by technology

For the two-dimensional convolutional neural network, feature extraction is performed on the frame, and only one frame of image is input into the two-dimensional convolutional network each time, and the obtained features lack the information of the time dimension

The three-dimensional convolutional neural network solves the deficiency of the time dimension of the two-dimensional convolutional neural network, but this time dimension lacks information exchange at different moments

Moreover, both of them are based on global features because the input is the entire picture or the entire video clip, and lack the description of local key information, the mutual influence between each local information, and the mutual influence between each local information and the overall information, such as human Information about the interaction with people and the description of information about the interaction between people and the environment

Images