Image multi-subtitle automatic generation method based on multiscale hierarchical residual network

Abstract

The invention discloses an image multi-subtitle automatic generation method based on a multiscale hierarchical residual network, and adopts an improved funnel network to capture multiscale target information. Firstly, when a funnel framework network is constructed, a densely connected polymerization residual block is put forward, and residual LSTM (Long Short Term Memory) is further put forward inorder to solve the problems of gradient vanishing and gradient explosion. By use of the method, high experiment performance is obtained, and the method has an obvious advantage on multi-subtitle taskacquisition.

Description

technical field [0001] The invention relates to a multi-subtitle acquisition technology, in particular to an image multi-subtitle automatic generation method based on a multi-scale layered residual network. Background technique [0002] Multi-caption acquisition is to obtain a non-fixed number of category target descriptors in an image. This work serves as a foundational service for many important applications, such as semantic image search, visual intelligence for chatbots, sharing images and videos on social media, helping people perceive the world around them, and more. [0003] The current study combines convolutional neural networks and recurrent neural networks to predict captions from image feature maps. However, some bottlenecks have been encountered in improving performance: 1) Target detection is still an open problem in computer vision; 2) From image feature space to description space is a nonlinear multimodal mapping; 3) Deeper network It is easier to learn thi...

AI Technical Summary

Problems solved by technology

[0008] Detection-based method: Although the sequence-based method achieves high accuracy on the subtitle acquisition task, it is often easy to ignore the details on the image, so a detection-based method is proposed to solve such problems

Images