Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Image semantic disambiguation method and device based on image and text semantic similarity

A technology of semantic similarity and similarity, applied in semantic analysis, character and pattern recognition, special data processing applications, etc., can solve problems such as image ambiguity, and achieve the effect of improving accuracy and reducing error rate

Active Publication Date: 2018-10-12
BEIJING JIAOTONG UNIV
View PDF8 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the image still has the problem of ambiguity

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image semantic disambiguation method and device based on image and text semantic similarity
  • Image semantic disambiguation method and device based on image and text semantic similarity
  • Image semantic disambiguation method and device based on image and text semantic similarity

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0053] The processing flow of an image semantic disambiguation method based on image and text semantic similarity provided by an embodiment of the present invention is as follows: figure 1 shown, including the following steps:

[0054] Step 1: Use the image saliency label to mark the image to be processed, obtain the label of the image to be processed, and mark the image content of the image to be processed.

[0055] Use a large number of known images to form training sample images, use the image visual saliency analysis method to perform saliency analysis on each training sample image, use NeuralTalk of convolutional neural network CNN, long short-term memory LSTM and / or recurrent neural network RNN The algorithm generates natural language descriptions for training sample images and obtains image saliency labels.

[0056] Collect a large number of images with polysemy ambiguity, such as images with apples, divided into Apple computers, mobile phones or edible apples, and put...

Embodiment 2

[0076] The structure of an image semantic disambiguation device based on image and text semantic similarity provided by this embodiment is as follows: image 3 As shown, the following modules are included:

[0077] The semantic processing module 31 is used to represent a meaning of a polysemy with a mean vector, and store all mean vectors and the meaning association of the polysemy corresponding to each mean vector in the mean vector database;

[0078] The image processing module 32 is used to mark the image to be processed using the image saliency label, obtain the label of the image to be processed, and mark the image content of the image to be processed, and convert the label and image content of the image to be processed into a vector form, to obtain the fusion vector of the image to be processed;

[0079] The image word sense disambiguation processing module 33 is used to use the cosine similarity to calculate the similarity between the fusion vector of the image to be p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an image semantic disambiguation method and device based on the image and text semantic similarity. The method comprises the steps that each meaning of a polysemous word is represented with a mean vector; a to-be-processed image is labeled with an image significance label to obtain a label of the to-be-processed image, and the label and image content of the to-be-processed image are converted into the form of vectors to obtain a fusion vector of the to-be-processed image; and the similarity between the fusion vector of the to-be-processed image and each mean vector is calculated by means of the cosine similarity, the mean vector with the highest similarity is found out, and the meaning corresponding to the mean vector with the highest similarity is determined as thecorrect interpretation of the to-be-processed image. According to the method, the image is converted into the vector by adopting the method of combining the image with the text, the image interpretation and image query ambiguity problems are solved, and effective image disambiguation is creatively achieved; and the image query and interpretation accuracy is greatly improved, and the error rate ofthe image interpretation is decreased.

Description

technical field [0001] The invention relates to the technical field of image semantic disambiguation, in particular to an image semantic disambiguation method and device based on image and text semantic similarity. Background technique [0002] Word sense disambiguation is a fundamental key research topic in the field of computational linguistics. As an "intermediate task", it is directly related to the efficiency and success of language processing application systems such as information retrieval, machine translation, text classification, and speech recognition. The word sense disambiguation of polysemous words is to solve the problem of labeling the meanings of homonymous words in different contexts in natural language. The universality of the distribution of polysemous words determines that the word sense disambiguation task of polysemous words must become one of the focuses of various application problems, such as machine translation, information retrieval, semantic anal...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06K9/46G06F17/27
CPCG06F40/30G06V10/462G06F18/22G06F18/253
Inventor 李浥东汪敏郎丛妍王涛冯松鹤董雅茹
Owner BEIJING JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com