Deep attention mechanism-based image description generation method

Abstract

The invention relates to a deep attention mechanism-based image description generation method. The method comprises a deep long-short-term memory network model building step of adding an attention mechanism function between units of a long-short-term memory network model and training a long-short-term memory network added with the attention mechanism function by utilizing features and descriptioninformation of training images extracted by a convolutional neural network to obtain a deep long-short-term memory network model, and an image description generation step of generating descriptions corresponding to the images through a convolutional neural network model and the deep long-short-term memory network model in sequence according to the images to be subjected to description generation.Compared with the prior art, the method has the advantages of high information extraction effectiveness, high deep expression capability, accurate description and the like.

Description

technical field [0001] The invention relates to the field of image understanding, in particular to an image description generation method based on a deep attention mechanism. Background technique [0002] Image caption generation is a very challenging task, and it has broad application prospects in the fields of early childhood education, visually impaired assistance, and human-computer interaction. It combines the two fields of natural language processing and computer vision to describe a natural image in the form of natural language, or translate the image into natural language. It first requires the system to be able to accurately understand the content in the image, such as identifying the scene in the image, various objects, object attributes, ongoing actions, and the relationship between objects, etc.; then according to grammatical rules and language structure, generate human understandable sentences. [0003] A variety of methods have been proposed to solve this pro...

AI Technical Summary

Problems solved by technology

However, these methods rely too much on the previous visual technology, the processing process is complicated, and the language model for generating sentences at the back end of the system is insufficiently optimized; when LSTM units are used to generate sentences, the model depth is relatively shallow (1-layer or 2-layer LSTM is often used), The semantic information of the generated sentences is not strong, and the overall performance is difficult to improve

Images