Group activity identification method based on coherence constraint graph long-short-term memory network

Abstract

The invention provides a group activity recognition method based on a coherence constraint graph long-short-term memory network, and the method comprises the following steps: (1), enabling the CNN features of all persons to serve as the input of the coherence constraint graph long-short-term memory network, and jointly learning the individual motion states of all persons along with time under thecoherence constraint of a space-time context; (2) quantifying contributions of related motions by learning attention factors corresponding to different motions by utilizing an attention mechanism of global context coherence; (3) aggregating all single motion states weighted by different attention factors into a hidden representation of the whole activity by adopting aggregation LSTM in each time step, and inputting the hidden representation of each activity into a softmax classifier; and (4) averaging the output of the softmax classifier of each time step to obtain a probability class vector of the group activity so as to deduce the class of the group activity.

Description

technical field [0001] The invention relates to an action recognition technology in the field of computer vision, in particular to a group activity recognition method based on a coherent constraint graph long short-term memory network. Background technique [0002] Traditional action recognition such as single-person recognition and two-person interaction usually consists of one person or two people appearing in a video, and such tasks have achieved satisfactory performance in the past few decades. Compared with traditional human behaviors, group activities are more complex but common behaviors in the scene. Unlike solo activities and two-person interactions, group activities are usually conducted by multiple people at the same time. Therefore, in group activity recognition, we need to model the behavior of multiple individuals and their interactions. This is a fine-grained recognition task, which is much more difficult than traditional single-person action recognition or ...

AI Technical Summary

Problems solved by technology

However, the above method assumes that the motions of all people contribute equally to the group activity, which suppresses the contribution of some coherent motions to the overall activity and exaggerates some outlier motions that are not related to the group activity.

Images