Multi-modal image retrieval method and system for appearance patent

A multi-modal image and image retrieval technology, applied in still image data retrieval, metadata still image retrieval, neural learning methods, etc., can solve the problems of low retrieval accuracy and low retrieval efficiency, achieve good retrieval effect, improve The effect of efficiency and accuracy

Active Publication Date: 2020-08-28
GUANGDONG UNIV OF TECH
View PDF4 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The present invention provides a multimodal image retrieval method and system for appearance patents to solve the problems of low retrieval efficiency and low retrieval accuracy in the existing multimodal image retrieval method for appearance patents

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal image retrieval method and system for appearance patent
  • Multi-modal image retrieval method and system for appearance patent
  • Multi-modal image retrieval method and system for appearance patent

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0051] A multimodal image retrieval method for appearance patents, such as figure 1 with 2 shown, including the following steps:

[0052] S1. Extract image features and text features of multiple views of appearance patents;

[0053] Among them, for the multi-view image features of the appearance patent, this embodiment uses the ResNet-based improved deep convolutional neural network Res2Net proposed by ShangHua Gao et al. in the 2019 CVPR paper to extract: z 1 ,z 2 ,…,z n , where n represents the number of views of the appearance patent, which may include left view, right view, front view, rear view, top view, three-dimensional figure 1 and three-dimensional figure 2 etc.; carry out weighted fusion to described image feature, obtain the image feature of multi-view fusion:

[0054]

[0055] i represents the i-th view of the design patent, and β represents the weight of the i-th view of the design patent. It should be noted that the weight ratio of the perspective view...

Embodiment 2

[0088] This embodiment provides a multimodal image retrieval system for appearance patents, such as Figure 4 shown, including:

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-modal image retrieval method and system for an appearance patent. Firstly, feature extraction and fusion are carried out on multiple views of an appearance patent, thenfeature extraction is carried out on a text, information of multiple modes is comprehensively considered, and finally deep visual semantic embedding is carried out, so that a good retrieval effect can be achieved in a large-scale appearance design patent database; for a tree structure in an ANN, compact coding representation is not performed on data so that efficiency is not high. Calculation ofthe Hamming distance in the hash method is not an accurate distance calculation problem. According to the invention, distance coding product quantization is provided, in the coding process, data points are coded into series connection of subspace clustering indexes, the distance between each data point and a reconstructed coded representation of the data point is coded, and an effective compact coded representation of each datum is formed; and therefore, the retrieval efficiency and accuracy are improved.

Description

technical field [0001] The invention relates to the technical field of image retrieval, in particular to a multimodal image retrieval method and system for appearance patents. Background technique [0002] Since images are the main content of design patents, the key technology for searching design patents is the core technology of image search. However, the design patent not only contains multiple view information of the patented design, but also contains related brief descriptions and other text information, such as the text description for the chair "This is a wooden rectangular dining table and chair with rounded corners", etc. Wait. Therefore, how to make good use of the text information of design patents for multimodal retrieval to optimize the retrieval effect is a problem of practical significance. [0003] In recent years, many scholars have invested in the multimodal learning technology because of its many modalities and rich information. However, how to integrat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/55G06F16/58G06F16/583G06K9/62G06N3/04G06N3/08
CPCG06F16/583G06F16/5866G06F16/55G06N3/08G06N3/045G06F18/23Y02D10/00
Inventor 叶街林杨志景谭俊鹏
Owner GUANGDONG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap