Image annotation method based on weak matching probability canonical correlation model

Abstract

The invention discloses an image annotation method and system based on a weak matching probability canonical correlation model, relating to the technical field of processing of network cross-media information. The image annotation method comprises the following steps: obtaining an annotated image and a non-annotated image in an image database, respectively extracting image features and textual features of the annotated image and the non-annotated image, and generating a matched sample set and an unmatched sample set, wherein the matched sample set contains an annotated image feature set and an annotated textual feature set; and the unmatched sample set contains a non-annotated image feature set and a non-annotated textual feature set; training the weak matching probability canonical correlation model according to the matched sample set and the unmatched sample set; and annotating an image to be annotated through the weak matching probability canonical correlation model. According to the invention, correlation between a visual modality and a textual modality is learned by using the annotated image, keywords of the annotated image and the non-annotated image simultaneously; and an unknown image can be accurately annotated.

Description

technical field [0001] The invention relates to the technical field of network cross-media information processing, in particular to an image labeling method based on a typical correlation model of weak matching probability. Background technique [0002] The Internet of Things and the Internet have rich multimedia information resources such as text, images, video, and audio. These information resources are heterogeneous, and it is difficult to directly find the correlation between them. Canonical correlation analysis (CCA) is a It is a statistical analysis tool used to analyze the correlation between two groups of random variables. Its correlation maintenance feature has been proved in theory. It is applied in the fields of economics, meteorology and genome data analysis. Construct the potential relationship between multimodal features, use a unified model to associate different types of multimodal data from the underlying features, and at the same time discover and maintain ...

AI Technical Summary

Problems solved by technology

[0003] In canonical correlation analysis, two groups of related random variables can come from multiple sources of information (such as a person's voice and image), or they can be different features extracted from the information of the same source (such as image color features and texture features) , but the training data must be strictly matched one-to-one. Many reasons make it difficult to obtain such strictly matched training data. For example, in a multi-sensor acquisition system, the sensor sampling frequency is not synchronized or the sensor failure will cause the data collected by different channels to be out of sync or The data of a certain channel is lost; single-modal data is relatively easy to obtain, but manual matching is very time-consuming and laborious. In practice, the multi-modal data often only has a small amount of one-to-one strict matching, and the rest of the large amount of data is not matched. Weakly matched multimodal data

[0004] There are two basic approaches to canonical correlation analysis for weakly matched multimodal data: discard unmatched data and only use canonical correlation analysis to process strictly matched multimodal data; match multimodal data according to specific criteria, but Neither approach is likely to achieve the desired result

Images