System and method for unsupervised joint visual concept learning based on image and text

Abstract

The invention discloses an unsupervised joint visual concept learning system and method based on image and text, including: a text analysis module, a cardinality example learning module and a multi-task clustering module, wherein: the text analysis module uses social media to analyze images Additional sentence description extracts corresponding noun as visual concept and cardinality word thereof as the additional constraint information of next module; Described cardinality example learning module utilizes cardinality-guided multi-example learning method to train the classifier of each visual concept; The task clustering module handles the diversity among concepts, that is, nouns referring to similar objects are aggregated into a large category as visual concepts. The invention utilizes unsupervised automatic learning to effectively solve the complex problem of manual calibration under large-scale data.

Description

technical field [0001] The invention relates to a visual concept method in the field of computer vision, in particular to an unsupervised joint visual concept learning system and method based on images and text. Background technique [0002] In the field of computer vision, traditional image classification and object detection methods rely more or less on human annotations, such as image-level or image-instance-level labels. In recent years, with the development of computer technology and the emergence of big data, large-scale visual concept learning has become an emerging research hotspot, and it is not easy to manually label millions or even tens of millions of data. Therefore, unsupervised learning is used for large-scale Learning visual concepts at scale is exactly what is needed today. [0003] Since it is particularly difficult to learn visual concepts from the picture itself, existing methods mostly rely on supervision or weak supervision. Existing visual concept le...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the defects in the prior art, the present invention provides an unsupervised joint visual concept learning system and method based on image and text, which can effectively solve the complex problem of manual calibration under large-scale data by using unsupervised automatic learning

Images