Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Image description method and device

An image description and image feature technology, applied in the field of image processing, can solve the problem of sentence deviation, and achieve the effect of accurate translation of sentences

Active Publication Date: 2019-10-08
BEIJING KINGSOFT DIGITAL ENTERTAINMENT CO LTD
View PDF6 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this case, the extraction of global information by the feature extraction model depends on the performance of the feature extraction model itself. Some feature extraction models will pay attention to one type of information in the image, and some feature extraction models will pay attention to another type of information in the image. One type of information, which will cause the translation model to often fail to use the complete global information corresponding to the image as a reference in the subsequent process, resulting in deviations in the output sentences

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image description method and device
  • Image description method and device
  • Image description method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar promotions without violating the connotation of the present application. Therefore, the present application is not limited by the specific implementation disclosed below.

[0066] The terminology used in one or more embodiments of this specification is for the purpose of describing a particular embodiment only and is not intended to limit the one or more embodiments of this specification. As used in the specification or embodiments and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and / or" as used in this specification in one...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an image description method and device, and the method comprises the steps: carrying out the feature extraction of a target image through employing a plurality of first featureextraction models, and obtaining the image features generated by each first feature extraction model; performing fusion processing on the image features generated by the plurality of first feature extraction models to generate global image features corresponding to the target image; performing feature extraction on the target image by using a second feature extraction model to obtain a target detection feature corresponding to the target image; inputting the global image feature and the target detection feature corresponding to the target image into a translation model, and using the generatedtranslation statement as a description statement of the target image, so that in the subsequent process of inputting the global image characteristics corresponding to the target image and the targetdetection characteristics into the translation model to generate the translation statement, the global image characteristics of the image information are richer as a reference, and the output translation statement is more accurate.

Description

technical field [0001] The present application relates to the technical field of image processing, and in particular, to a method and apparatus for image description, a computing device, and a computer-readable storage medium. Background technique [0002] Image description is to automatically generate a descriptive text based on the image, similar to "seeing the picture and talking". Image captioning is simple and natural for humans, but challenging for machines. The reason is that the machine not only needs to be able to detect the objects in the image, but also to understand the relationship between the objects, and finally to express it in a reasonable language. [0003] In the prior art, the process of image description requires a machine to extract local information and global information from the target image, input the global information and local information to the translation model, and use the sentence output by the translation model as the description informatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/46G06K9/62
CPCG06V10/457G06V10/44G06V2201/07G06F18/2414G06F18/253G06V10/82G06V20/70G06V10/806G06F40/40G06V10/424
Inventor 宋振旗李长亮廖敏鹏
Owner BEIJING KINGSOFT DIGITAL ENTERTAINMENT CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products