Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A video behavior recognition method based on spatio-temporal fusion features and attention mechanism

A technology of time-space fusion and recognition method, which is applied in character and pattern recognition, computer parts, instruments, etc., can solve problems such as inability to deal with sequence problems, and achieve the effect of improving accuracy

Active Publication Date: 2018-12-28
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF10 Cites 125 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Although convolutional neural networks can extract spatial features of videos, they cannot handle sequence problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A video behavior recognition method based on spatio-temporal fusion features and attention mechanism
  • A video behavior recognition method based on spatio-temporal fusion features and attention mechanism
  • A video behavior recognition method based on spatio-temporal fusion features and attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0050] For the convenience of description, the relevant technical terms appearing in the specific implementation are explained first:

[0051] LSTM (Long Short-Term Memory): long short-term memory network;

[0052] figure 1 It is a flow chart of the video behavior recognition method based on spatio-temporal fusion features and attention mechanism of the present invention.

[0053] In this example,

[0054] The LSVRC2012 data set is used for the pre-training of the Inception V3 network, and the HMDB-51 and UCF-101 data sets are used for model simulation and verification analysis.

[0055] The HMDB-51 data set contains 6849 videos, the video content is mainly from movie clips, and is divided into 51 categories, of which 5222 are used as training sets, 300 are used as verification sets, and 1327 are used as test sets.

[0056] The UCF-101 dataset is a video action recognition dataset collected from real life. The video content is all from YouTube videos, including 13,320 video...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a video behavior identification method based on spatio-temporal fusion characteristics and attention mechanism, The spatio-temporal fusion feature of input video is extracted by convolution neural network Inception V3, and then combining with the attention mechanism of human visual system on the basis of spatio-temporal fusion characteristics, the network can automaticallyallocate weights according to the video content, extract the key frames in the video frame sequence, and identify the behavior from the video as a whole. Thus, the interference of redundant information on the identification is eliminated, and the accuracy of the video behavior identification is improved.

Description

technical field [0001] The invention belongs to the technical field of behavior recognition, and more specifically relates to a video behavior recognition method based on spatio-temporal fusion features and an attention mechanism. Background technique [0002] Research related to behavior recognition is increasingly widely cited in many application scenarios, such as security monitoring, autonomous driving, video retrieval, etc. Behavior recognition generally refers to identifying the behavior of individuals or groups from video sequences. Often specific actions occur over a sequence of contiguous video frames, not just a single video frame. Therefore, the motion information in video is very important for behavior recognition, how to effectively characterize the spatio-temporal features in video is a hot spot in the field of behavior recognition research. [0003] Traditional behavior recognition relies on manual features extracted from video frame sequences and optical fl...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06N3/04
CPCG06V40/20G06V20/41G06V20/46G06N3/045
Inventor 徐杰余兴盛纾纬魏浩亮
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com