Visual and auditory perception integrated multitask collaborative identification method and system

Abstract

The invention provides a visual and auditory perception integrated multitask collaborative identification method and system and belongs to the technical field of multi-source heterogeneous data processing and identification. The system comprises a universal feature extraction, a collaborative feature learning module and a suitable scene feedback, assessment and identification module. Based on timesynchronization matching mechanism of multi-source heterogeneous data, universal features of the multi-source heterogeneous data are extracted; a long-time dependent memory model is established, anduniversal features serving as priori knowledges are continuously learned by cooperating with a collaborative attention mechanism based on external dependence; environmental perception parameters in the multi-source heterogeneous data are extracted, a progressive network depth collaborative reinforcement identification mechanism is established, and multitask identification is achieved according tothe learning features and task demands of the memory model. The system combines with a suitable scene computing theory based on environmental perception, judges the weight of each identified task through depth reinforcement feedback, self-adaptively adjusts the priority of each task according to environmental change and achieves the effect of simultaneously outputting multiple visual and auditoryperception identification results.

Description

technical field [0001] The invention relates to the technical field of multi-source heterogeneous data processing and recognition, and in particular to a multi-task cooperative recognition method and system that integrates audio-visual perception. Background technique [0002] After experiencing ups and downs for a period of six years, artificial intelligence is about to enter a full-scale explosion by taking advantage of the wave of the IT era such as the Internet, mobile Internet, and Internet of Things, based on deep neural network algorithms, and supported by big data, cloud computing, and smart terminals. new era. The continuous growth of communication bandwidth and the continuous improvement of transmission speed have rapidly lowered the threshold for obtaining massive audio / video data. Faced with the urgent need for ultra-high-speed, mobile and universal storage and processing of massive data, traditionally weak artificial intelligence based on single-mode single-tas...

AI Technical Summary

Problems solved by technology

[0006] Deep neural network training requires a large amount of training data, which makes it powerless for small-scale data tasks. Faced with the high training and labeling costs of massive data, it has poor performance for real recognition tasks with continuous data stream input.

[0007] The deep neural network model is complex, the number of parameters is huge, and the training process requires powerful computing facilities. At the same time, when facing different recognition tasks, different convolutional layer structures are used, which makes it difficult to achieve rapid and balanced allocation of network resources.

Images