Network live video feature extraction method in complex scene based on joint attention ResNeSt

Abstract

The invention relates to a network live video feature extraction method in a complex scene based on joint attention ResNeSt. Firstly, key frame extraction is performed on a network live video to obtain key frame data of the video; in order to utilize multi-scale features of video frames, a parallel path is designed according to a multi-scale structure of a feature pyramid network. The parallel path is constructed from bottom to top, information exchange is carried out between the parallel path and an original main path by utilizing transverse connection and oblique connection, and the transverse connection and the oblique connection are convolution operations. Considering that the picture representation form of network live broadcast is mostly a human main body, and a large amount of redundant information is mingled, the space-channel joint attention is introduced, and the picture main body characteristics are conveniently focused. And finally, a ResNeSt feature extraction module is constructed by combining the parallel feature pyramid fused with the joint attention with a convolution layer and a pooling layer, and feature extraction of the network live video in a complex scene is realized through superposition of multiple layers of modules.

Description

technical field [0001] The present invention takes network live video in complex scenes as the research object, and extracts features of live video through joint attention and ResNeSt network, thereby forming an efficient feature expression of live video. First, use the parallel feature pyramid to perform feature convolution on the key frame of the video; in the convolution process of the feature pyramid, obtain the low-level visual information and high-level semantic information of the video by introducing a joint attention mechanism; finally combine the split attention residual network (ResidualNetworks with Split-Attention, ResNeSt), forming an efficient feature expression for live webcast videos. Background technique [0002] With the advent of the Internet self-media era, more and more people begin to share their lives on the Internet in the form of live videos, and the number of live videos on the Internet is also increasing geometrically. Webcasting has a strong abil...

AI Technical Summary

Problems solved by technology

However, when the video scene is relatively single, the number of people is small, and the edge of the object is clear, using the existing deep learning network can obtain better performance, but when there are many types of scenes, the number of people is uncertain, and the lighting conditions are affected In complex live video scenes under limited conditions, directly applying the above-mentioned deep network is not easy to effectively learn spatio-temporal context information, which affects the improvement of accuracy

Images