Image content automatic description method constructed by Chinese visual vocabulary

A technology of visual vocabulary and image content, applied in the field of image semantic understanding, which can solve the problems of less image description data, complex sentence structure, and poor quality.

Active Publication Date: 2020-08-25
CAPITAL NORMAL UNIVERSITY
View PDF6 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The reason is that on the one hand, Chinese image description data is less and the quality is poor, which limits the development of automatic image content generation; on the other hand, Chinese words have rich meanings and complex sentence structures, and there are also difficulties in semantic understanding.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image content automatic description method constructed by Chinese visual vocabulary
  • Image content automatic description method constructed by Chinese visual vocabulary
  • Image content automatic description method constructed by Chinese visual vocabulary

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] Below at first the present situation of the image automatic generation description technology involved in the present invention is analyzed:

[0022] The current image automatic description generation methods can be summarized into three categories, namely template-based methods, similarity retrieval-based methods, and deep learning-based methods.

[0023] Thanks to the development of image object recognition technology, researchers have proposed template-based image description generation methods. Specifically, the object and its attribute information in the image are detected through target recognition, and then the information is embedded in a pre-designed template in an appropriate manner. In 2010, Farhadi et al. used detectors to detect objects in images to infer triples, and used templates to convert them into descriptive text. In 2011, Yang et al. used Hidden Markov Model to select possible objects, verbs, prepositions and scene types to fill sentence tem...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an image content automatic description method constructed by a Chinese visual vocabulary. The method comprises the following steps of: a, performing word segmentation processing on a plurality of description sentences corresponding to a single picture by using a Chinese word segmentation tool, selectively reserving nouns, verbs and adjectives in a word list according to statistical word frequencies, and forming a Chinese visual vocabulary by using the reserved words; b, predicting the Chinese visual vocabulary based on a Chinese vocabulary prediction network to obtainimage annotation information; c, based on the automatic image description model, extracting image convolution features by using an encoder, and decoding the image convolution features as initial inputinto Chinese description statements by using a decoder; image annotation information can be obtained by predicting the image vocabulary through the vocabulary prediction network, a residual structureis added to the Chinese visual vocabulary prediction network, and the problem that the number of layers of the Chinese visual vocabulary prediction network is deepened can be effectively solved.

Description

technical field [0001] The invention relates to image semantic understanding technology, and specifically provides an automatic description method for image content constructed by a multi-channel Chinese visual vocabulary. Background technique [0002] Image semantic understanding technology combines two research directions of computer vision and natural language processing. It is a research hotspot in the field of artificial intelligence and an effective method to reduce the semantic gap between low-level features and high-level semantics of images. Image semantic understanding technology provides machines with the ability to process multi-modal data, which can effectively reduce the semantic gap between low-level features and high-level semantics of images. Its core technology is to combine the relevant knowledge of computer vision and natural language processing. The content of the text is analyzed and understood, and feedback is given in the form of text semantic informa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F40/284G06F40/216G06K9/46G06K9/62G06N3/04G06N3/08
CPCG06F40/284G06F40/216G06N3/049G06N3/08G06V10/40G06N3/045G06F18/24
Inventor 张凯周建设刘杰吕学强
Owner CAPITAL NORMAL UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap