Zero sample classification method based on cross-modal embedding of manifold constraint

Abstract

The invention discloses a zero sample classification method based on cross-modal embedding of a manifold constraint. The method comprises the steps of learning a conversion matrix of mapping from a visual space to a category semantic space by utilizing a cross-modal embedding model of the manifold constraint; mapping test samples to the category semantic space from the visual space by utilizing the learned conversion matrix to obtain embedding vectors of the test samples in the category semantic space; and calculating relationships between the embedding vectors of the test samples in the category semantic space and semantic features of test categories by utilizing a Euclidean distance, and classifying the test samples by utilizing a nearest neighbor classifier. According to the zero sample classification method based on the cross-modal embedding of the manifold constraint, semantic information among different modes and judgment information among different categories can be better mined, and local structure in the mode conversion process is kept by utilizing the manifold constraint.

Description

technical field [0001] The invention relates to a zero-sample classification method. In particular, it concerns a zero-shot classification method based on manifold-constrained cross-modal embeddings. Background technique [0002] In traditional object classification techniques, the categories to which the test samples belong must be included in the categories that have appeared in the training phase. In order to obtain better classification results, a large number of labeled samples are required for each training category. However, labeling samples requires a lot of time and manpower, and it is difficult to obtain labeled samples for some categories. In order to solve the problem of missing training samples, zero-shot classification has attracted the attention of researchers. [0003] The task of zero-shot classification is to classify test samples into categories that did not appear in the training phase. Traditional classification techniques cannot solve this problem. ...

AI Technical Summary

Problems solved by technology

However, current cross-modal embedding frameworks only focus on the semantic alignment between different modal spaces, while ignoring the local structural changes during the cross-modal process.

Images