Video ucl semantic indexing method and device based on deep learning

Abstract

The invention discloses a UCL semantic indexing method and device based on deep learning. First, the neural network is used to extract the low-level semantic features of the video; then, based on the flexible sampling of features and the attention mechanism, the video natural language description generation model S2VT is improved to generate the S2VT-FFSA model, which takes the low-level semantic features of the video as input and outputs the video Natural language description features, combined with voice natural language description features to generate high-level semantic features such as video keywords, solves the problem of insufficient semantic feature extraction to a certain extent; finally, using UCL to index rich semantic features, propose video content The UCL indexing method makes video indexing more standardized. The invention can not only extract rich semantic features of videos accurately, but also index these features objectively and standardizedly.

Description

technical field [0001] The invention relates to a video UCL semantic indexing method and device based on deep learning, which uses deep learning technology to automatically extract video low-level features and high-level features, and based on the UCL national standard GB / T35304-2017 to index video semantic features, belonging to the Internet technology field. Background technique [0002] With the rapid development of computer technology and information technology, video production methods and upload channels have become increasingly convenient, resulting in the existence of massive video resources on the Internet. In order to solve the problem of video information overload, major video portals will provide users with video search and recommendations. In order to effectively manage video resources and efficiently realize the above functions, it is particularly important to accurately extract video features and standardize indexing. However, the following problems still ex...

AI Technical Summary

Problems solved by technology

However, the following problems still exist in the current video feature extraction and indexing: First, the traditional video feature extraction method does not perform well in the high-level semantic extraction of videos. If the video semantic features are extracted artificially, the time efficiency is not high, and As a result, there are often differences in semantic features due to different personal opinions, and it is difficult to generate high-level semantic features of videos under a unified standard framework; secondly, video semantic extraction methods based on deep learning still need to be improved in terms of description accuracy, for example , S2VT (Sequence to Sequence-Video to Text) video natural language description model (Venugopalan S, et al. Sequence to Sequence--Video to Text [C]. IEEE International Conference on Computer Vision (ICCV). IEEE, 2015) for surrounding videos Fragments pay less attention, and the input method of its model is too single; finally, the multimedia content description interface MPEG-7 can index video features, but it can only standardize visual features such as color, texture, shape, and some semantics features, and other semantic features require the user to define a new description mode

If there is no unified video content coding specification, the system needs to design a method for obtaining specified video features for different coding formats. The universality of video features in various recommendation systems cannot be guaranteed, and it creates difficulties for the unified management of video resources.

Images