Attention mechanism-based natural language generating method for remote sensing images

Abstract

The invention discloses an attention mechanism-based natural language generating method for remote sensing images, and belongs to the technical field of computer vision. A CNN part comprises a commonconvolutional pooling layer, a C-S layer and a full connection layer; an RNN part structurally comprises a multi-layer model unit, a GRU unit and an attention unit; the remote sensing images at different moments in the same area are input into the initialized CNN part to obtain eigenvectors; the eigenvectors are input into the GRU unit and the attention unit, the attention unit also receives hidden layer states from the GRU unit, and the attention unit maps the eigenvectors and the hidden layer states to low dimension for further compression and normalization to obtain eigenvectors subjected to weighted average; and the eigenvectors subjected to the weighted average and the hidden layer states of the GRU unit are integrated through the multi-layer model unit, and then maximum likelihood isperformed through a hidden layer and a normalization layer to obtain an integer sequence. The problem that a result of generating a natural language by the remote sensing images at present is not ideal is solved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a method for generating natural language of remote sensing images based on an attention mechanism. Background technique [0002] With the development and popularization of artificial intelligence, image annotation, as the integration of computer vision and natural language processing, has become more and more important. It is used in many important fields such as image retrieval, children's education and life assistance for visually impaired people. have important applications. A method that combines deep convolutional neural networks and recurrent neural networks has achieved remarkable progress on the problem of image annotation. However, the achievements in the field of computer vision in some areas of remote sensing images, image annotation has not well realized its potential. For remote sensing images, the classic combination of CNN and RNN in the field...

AI Technical Summary

Problems solved by technology

[0009] The purpose of the present invention is to solve the following problems: (1) remote sensing images are different from natural images, they contain a large amount of irrelevant information, and have the characteristics of scale fuzzy, classification ambiguity and rotation ambiguity, and low density, which may lead to remote sensing images. Objects show different semantics at different scales, and it is difficult to describe the fusion area in remote sensing images, which can easily lead to ambiguity in classification; (2) the data set of remote sensing images is limited and the trained model is easy to overfit; (3) Neglecting the low-density characteristics of the remote sensing image itself and the characteristics of multiple images in the same area at different times may result in only a small area containing a small effective area in a large remote sensing image, making the model unable to fully describe the effective area. information; the present invention provides a method for generating natural language based on attention mechanism

Images