A Video Understanding Method Based on Compression-Excitation Pseudo-3D Network

Abstract

The present invention provides a video comprehension method based on compression-excitation pseudo-three-dimensional network, the method comprising: preprocessing training data and test data to form a training set and a test set; The difference network; the test set is used to test the pseudo-three-dimensional residual network based on the compression-incentive mechanism; and the detailed structure of the pseudo-three-dimensional residual network based on the compression-incentive mechanism is given. A video comprehension method based on the compression-excitation pseudo-3D network proposed by the present invention evenly extracts the spatial and temporal features of the input video clips, reduces the amount of parameters compared with the 3D convolution model, and deepens the number of network layers. Deeper features are extracted; and the interdependence between feature channels is explicitly modeled, thereby improving network performance; the prediction results of the test samples are averaged as the final prediction results, which increases the accuracy and robustness of the results sex.

Description

technical field [0001] The invention belongs to the technical field of computer vision, relates to the field of video comprehension and classification, and in particular relates to a video comprehension method based on a compression-excitation pseudo three-dimensional network. Background technique [0002] A large amount of image and video data is being generated every minute, which also promotes the development of multimedia content understanding applications such as search and recommendation. How to extract video features well is of great significance for video content analysis and understanding. In the image field, the integration of the residual network model has been able to achieve a 3.57% top-5 error rate (error) on the ImageNet dataset (a large-scale visualization database for visual object recognition software research), which is better than the error rate. The rate is 5.1% human level. Compared with images, video contains complex timing information in addition to ...

AI Technical Summary

Problems solved by technology

[0006] 3) The three-dimensional convolutional model has a larger number of parameters than the two-dimensional convolutional network, so training is very difficult, so most of them use a shallow structure, which makes it difficult to extract deep features

[0007] In addition, the current basic convolutional neural network of the video understanding classification model has some problems: the convolution kernel, as the core of the convolutional neural network, usually aggregates the spatial information and the information of the feature dimension on the local receptive field and finally obtains it. global information

The convolutional neural network consists of a series of convolutional layers, nonlinear layers, and subsampling layers, so that they can capture the characteristics of the image from the global receptive field for image description, but it is quite difficult to learn a very powerful network. difficult

Images