Fine-grained visual question-answering method combined with multi-view attention mechanism

Abstract

The invention relates to a fine-grained visual question-answering method combined with a multi-view attention mechanism. The guiding effect of specific semantics of the problem is fully considered. Amulti-view attention model is provided. A plurality of salient target areas related to a current task target (problem) can be effectively selected From multiple perspectives, region information related to answers is acquired in images and question texts, regional significance features are extracted in the images under the guidance of question semantics. The characteristic expression of finer granularity is realized; the multi-view attention model has the advantages that the multi-view attention model is constructed, the situation that a plurality of important semantic expression areas exist in the image is expressed, the depicting capacity is high, the effectiveness and comprehensiveness of the multi-view attention model are improved, and therefore the semantic relevance of image area significant features and question features is effectively enhanced, and the accuracy and comprehensiveness of semantic understanding of visual questions and answers are improved. The visual question-answering task is carried out by adopting the method, the steps are simple, the efficiency is high, the accuracy is high, the method can be completely used for business, and the market prospect is good.

Description

technical field [0001] The invention relates to the technical fields of computer vision and natural language processing, and more specifically, relates to a fine-grained visual question answering method combined with a multi-view attention mechanism. Background technique [0002] With the rapid development of computer vision and natural language processing, visual question answering system has become one of the more and more popular research fields of artificial intelligence. Visual question answering technology is an emerging topic. Its task is to combine the two disciplines of computer vision and natural language processing, take a given image and natural language questions related to the image as input, and generate a natural language answer as output. Visual question answering is a key application direction in the field of artificial intelligence. By simulating real-world scenarios, visual question answering can help users with visual impairments perform real-time human-...

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to overcome the deficiencies of the prior art, and provide a fine-grained visual question answering method combined with a multi-view attention mechanism, which can effectively improve the extraction of visual semantic information accuracy and comprehensiveness, and reduce the influence of redundant data and noise data, thereby improving the fine-grained recognition ability of the visual question answering system and the judgment of complex problems, and improving the accuracy of the visual question answering system and the reliability of the model to a certain extent explanatory

Images