Image processing method and system

Abstract

The embodiment of the invention discloses an image processing method. The method comprises: extracting image CNN (convolutional neural network) features of a target image through a CNN, and generating semantic text features corresponding to the target image; extracting first spatial structure information from the semantic text features; and analyzing the image CNN features according to the spatial structure information to acquire second spatial structure CNN features which is in the image CNN features and corresponds to the first spatial structure information. The invention also discloses an image processing system. By adopting the method and the system, the spatial structure information can be obtained directly from the image CNN features, thus people are helped to further understand the CNN features, and applications such as image and text cross-retrieval, image annotation, object detection, zero-shot learning and visual Q&A systems can be enabled to be benefited.

Description

technical field [0001] The invention relates to the field of computers, in particular to an image processing method and system. Background technique [0002] In recent years, in the intersection of image and semantic text, such as zero-shot learning (Zero-ShotLearning), image description generation (Image to Text), visual question answering system (Visual Q&A), there have been many impressive new methods and excellent work. [0003] Image annotation techniques can automatically generate textual descriptions of images. Using Convolutional Neural Networks (CNN) to extract image features, map the image features to a subspace constructed by the image features and the features of this paper, obtain the mapped subspace features, and then use the long short-term memory (Long Short-Term Memory) Term Memory, LSTM) model as a decoder, converts the mapped subspace features into text features, and further into natural language. [0004] The visual question answering system is a work ...

AI Technical Summary

Problems solved by technology

CNN has a powerful ability to represent images, but at present people do not fully understand what CNN features are. The use of CNN in the above technologies is limited to extracting image features, that is to say, CNN is used as a It is used as a "black box"; if the information in the CNN features is not fully studied and understood, it will bring great inconvenience to the follow-up work. For example, in the regional convolutional neural network RCNN network, it is necessary to first extract a lot of images Small image blocks, and then extract CNN features for each image block, this process is very time-consuming

Images