Image-text matching method and device based on artificial intelligence, equipment and storage medium

Abstract

The invention provides an image-text matching method and device based on artificial intelligence, an electronic device and a storage medium, and the method comprises the steps: obtaining the semanticword features of all words in a given text, and synthesizing the global features of the given text based on the word features; acquiring regional position features of the candidate image for representing a position relationship in the region; acquiring regional visual features of each region in the candidate image; combining the regional visual features and the regional position features to form regional features corresponding to each region of the candidate image, and obtaining global features corresponding to the candidate image; and determining similarity scores of the candidate images andthe given text based on the word features, the global features of the given text, the regional features and the global features of the candidate images, and determining that the candidate images are matched with the given text when the similarity scores are greater than a similarity score threshold. Through the method, image-text matching accuracy can be improved.

Description

technical field [0001] The present invention relates to artificial intelligence technology, in particular to an artificial intelligence-based image-text matching method, device, equipment and storage medium. Background technique [0002] Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. [0003] Deep learning (DL, Deep Learning) is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Specializes in the study of how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Deep learning typical...

AI Technical Summary

Problems solved by technology

[0004] With the development of Internet technology, in more and more scenarios, it is necessary to search for images based on text to obtain images that match the content represented by the text or to search for text based on images to obtain text that matches the content represented by the image. In the technology, the task of image selection or text selection is realized through deep learning technology, but the inventors found that the related technology ignores the influence of the positional relationship in the image area on the content expressed by the image, and the global information of the image text on the image and text. The impact of expression

Images