Graph representation learning framework based on specific semantics and multi-label classification method thereof

Abstract

The invention discloses a graph representation learning framework based on specific semantics and a multi-label classification method thereof. The framework comprises a semantic coupling module used for extracting image features from an input image by using a convolutional neural network, combining the image features with semantic features, introducing an attention mechanism, guiding learning of image feature weights by using the semantic features, and acting on the image features to obtain new feature vectors; a semantic interaction module which is used for constructing a large knowledge graph by constructing the relevance of category coexistence in the knowledge graph statistical data set, then performing feature expression on the knowledge graph by utilizing a door graph network, and iteratively updating the knowledge graph to obtain feature representation of the knowledge graph; and a knowledge embedding expression module which is used for combining the feature representation learned by the knowledge expression of the semantic interaction module with the image feature learning extracted by the semantic coupling module so as to realize multi-label classification.

Description

technical field [0001] The invention relates to the technical field of machine learning, in particular to a graph representation learning framework based on specific semantics and a multi-label classification method thereof. Background technique [0002] Image classification tasks often occur in daily life. It is to distinguish different types of images according to the semantic information of images. basis of the task. [0003] Multi-label image classification is a fundamental but practical task in computer vision, since real-world images often contain multiple distinct semantic objects. Currently, it is receiving increasing attention because it supports a large number of key applications in content-based image retrieval and recommendation systems. Besides the challenges of dealing with complex variations in angle, scale, occlusion, lighting, predicting the presence of multiple labels also requires mining semantic object regions as well as modeling associations and intera...

AI Technical Summary

Problems solved by technology

However, object localization techniques, which search numerous category-agnostic and redundant proposals, are difficult to be integrated into deep neural networks for end-to-end training, while visual attention networks only roughly localize object regions due to lack of supervision or guidance.

[0005] At present, although RNN (Recurrent Neural Network, recurrent neural network) / LSTM (Long ShortTerm Memory Network, long short-term memory network), further simulates the context dependence between semantic regions and captures label dependencies, however, RNN / LSTM sequentially Model region / label dependency, which cannot fully exploit this property, since there is a direct correlation between each region or label pair, moreover, they do not explicitly model statistical label co-occurrence, which is why the present invention helps multi-label image classification The essential

Images