Visual behavior recognition method and system based on text semantic supervision and computer readable medium

Abstract

The invention discloses a visual behavior recognition method and system based on text semantic supervision and a computer readable medium. The method comprises the steps of text semantic feature extraction, visual feature extraction based on text semantic supervision and visual behavior recognition construction. According to the method, text description normal forms of various behaviors are concluded on the basis of text description sentences of a video sample set of the same type of behaviors, a sample pair data set is constructed, and action semantic feature vectors and relation semantic feature vectors of the text description sentences are extracted from a text semantic extraction model; the extracted action visual feature vector and the relation visual feature vector are supervised byusing the action semantic feature vector and the relation semantic feature vector, and behavior recognition is performed by using the extracted action visual feature vector and the relation visual feature vector; therefore, the problems that in the current visual behavior recognition field, the accuracy of visual behavior recognition is not high, the efficiency of text semantic supervision is nothigh, and actions and relations between behaviors cannot be accurately recognized are solved.

Description

technical field [0001] The invention belongs to the technical fields of natural language processing, computer vision and behavior recognition, and relates to a visual behavior recognition method, system and computer-readable medium based on text semantic supervision. Background technique [0002] Artificial intelligence, especially computer vision technology, has penetrated into and affected all aspects of people's lives, and is inseparable from people's lives, such as human-computer interaction, object positioning, intelligent robots and other fields are closely related to behavior recognition technology in computer vision. The problem of self-visual behavior recognition has shown great economic value and application prospects, and has become a hot topic in the field of computer vision research, favored by many scholars. [0003] The research on visual behavior recognition can be traced back to 1975. At that time, the visual behavior recognition was mainly based on human be...

AI Technical Summary

Problems solved by technology

[0005] In order to achieve the above-mentioned purpose, the embodiment of the present invention provides a visual behavior recognition method, system and computer-readable medium based on text semantic supervision, and summarizes the text description paradigm of various behaviors based on the text description sentences of the video sample set of the same type of behavior , by randomly selecting text description sentences and / or text description paradigms to construct a sample pair data set, train the text semantic extraction model on the sample pair data set, extract the action semantic feature vector and relational semantic feature vector of the text description sentence, use the action semantic Feature vectors and relational semantic feature vectors supervise the action visual feature vectors and relational visual feature vectors extracted from the visual feature extraction model, and use the extracted action visual feature vectors and relational visual feature vectors for behavior recognition to solve the current visual behavior In the field of recognition, the accuracy of visual behavior recognition is not high, the efficiency of text semantic supervision is not high, and the actions and relationships between behaviors cannot be accurately recognized

Images