A Knowledge Base Completion Method Based on Multimodal Representation Learning

Abstract

The invention relates to a knowledge base completion method based on multi-modal representation learning. Given a knowledge base KB, the KB includes two parts, one is a known knowledge set, and the other is an unknown knowledge set; Data preprocessing of the data; a knowledge base completion model ConvAt is proposed, and the multimodal representation of the head entity and tail entity is first generated for the acquired data; then the multimodal representation of the head entity, the structural feature vector of the relationship and the tail entity The multi-modal representation of is concatenated by columns, processed by the convolutional neural network module, channel attention module and spatial attention module respectively, and finally multiplied with a weight matrix to obtain the score of the triplet (h, r, t); Use the loss function to train the completion model in step S2, and use the trained model to complete the knowledge base. The algorithm proposed by the invention can fuse external information and utilize richer semantic information.

Description

technical field [0001] The invention relates to the field of knowledge base completion, in particular to a knowledge base completion method based on multimodal representation learning. Background technique [0002] A variety of knowledge base completion methods have emerged in recent years, among which methods based on knowledge representation learning are currently an active research field of knowledge base completion. A key problem in representation learning is to learn low-dimensional distributed embeddings of entities and relations. [0003] At present, there are mainly two kinds of information that can be used for knowledge representation learning. The first is the triples that already exist in the knowledge graph. It mainly includes: knowledge graph representation learning methods based on translation / translation, such as TransE; methods based on tensor / matrix decomposition, such as RESCAL model; representation learning models based on neural network, such as ConvE. ...

AI Technical Summary

Problems solved by technology

Most of the existing methods for integrating external information only consider a single modal information, most of which are text modals, and fail to use the complementary characteristics between multiple modalities to learn more comprehensive features.

[0004] At present, most knowledge graph representation learning only considers the structural knowledge between entities and relationships. The performance of this type of model is limited by the display and storage of knowledge, resulting in unstable knowledge base completion. Possess knowledge of multiple modes, such as text, picture, audio and video, etc.

These external knowledge of different modalities can enrich and expand the existing knowledge base to a certain extent, and then provide richer semantic information for downstream tasks such as question answering and link prediction; existing representation learning methods that incorporate external information , most of them only consider a single modality information, and fail to use the complementary characteristics between multimodalities to learn more comprehensive features

Images