Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Behavior identification method based on long-time deep time-space network

A space-time network and recognition method technology, applied in the field of image recognition, can solve problems such as difficulties in data collection and annotation, limited size and diversity, etc., and achieve the effect of improving recognition rate and robustness

Inactive Publication Date: 2018-07-24
NANJING UNIV OF POSTS & TELECOMM
View PDF3 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, publicly available action recognition datasets (e.g., UCF101, HMDB51) are still limited in size and diversity due to difficulties in data collection and annotation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Behavior identification method based on long-time deep time-space network
  • Behavior identification method based on long-time deep time-space network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0025] see figure 1 , a behavior recognition method based on a long-term deep spatio-temporal network, including the following steps:

[0026] S1. Build a multi-channel feature splicing network MFCN (Multi-Chunnel Feature Connected Network) model;

[0027] S2, select the video behavior data set, extract the video frame and optical flow frame of each video in the video behavior data set, and use the collection of video frames as the color image sequence data set I rgb , a collection of optical flow frames as an optical flow image sequence dataset I flowx , I flowy ;

[0028] S3, the color image sequence data set I rgb and Optical Flow Image Sequence Dataset I flowx , I flowy According to the continuous multi-frames, it is divided into several segments, and the segments are input into the multi-channel feature splicing network model. First, the spatio-temporal features of the continuous frames of each segment are extracted through the low-level convolutional layer, and the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a behavior identification method based on a long-time deep time-space network. The method comprises the following steps: constructing an MCFCN (multi-channel feature connected network) model; selecting a video behavioral data set, and extracting a color image sequence data set and an optical flow image sequence data set of each video in the video behavioral data set; and dividing the color image sequence data set and the optical flow image sequence data set into a plurality of fragments according to successive multiple frames, inputting the fragments into the MCFCN model, extracting spatial-temporal features of each fragment successive frame at the lower layer, then, through middle-layer splicing, generating overall spatial-temporal features of each video fragment, connecting the overall spatial-temporal features of the video fragments into overall spatial-temporal features of the video according to fragment sequence, then, fusing the overall spatial-temporal features of the video at the higher layer and finally, outputting the classification result of the video behavior through a softmax layer. Complex behaviors in the video are identified by extracting thespatial-temporal features in a long-time multi-frame image sequence, thereby improving video complex behavior recognition rate and robustness.

Description

technical field [0001] The invention belongs to the technical field of image recognition, and in particular relates to a behavior recognition method based on a long-term deep spatio-temporal network. Background technique [0002] Video-based behavior recognition is widely used in many fields such as security and behavior analysis. In the field of action recognition, there are two key and complementary aspects: appearance and dynamics. The performance of a recognition system largely depends on the ability to extract and utilize relevant information from it. However, extracting such information is difficult due to many complexities such as scale changes, viewpoint changes, and camera motions. Therefore, it becomes crucial to design effective features that can address these challenges while preserving categorical information for behavioral categories. Recently, Convolutional Networks (ConvNets) have achieved great success in classifying images of objects, scenes, and complex...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06N3/04
CPCG06V20/41G06V20/46G06N3/045
Inventor 孙宁宦睿智李晓飞
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com