Cross-modal retrieval algorithm based on text concept extension

A retrieval algorithm and cross-modal technology, applied in the field of cross-modal retrieval, can solve problems such as the imbalance between the amount of video information and text information, and the reduction of retrieval performance.

Pending Publication Date: 2022-08-09
镇江智栎高科技有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a cross-modal retrieval algorithm based on text concept expansion, aiming to solve the problem of unbalanced video information and text information, which reduces retrieval performance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal retrieval algorithm based on text concept extension
  • Cross-modal retrieval algorithm based on text concept extension
  • Cross-modal retrieval algorithm based on text concept extension

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

[0038] see Figure 1 to Figure 5 , the present invention provides a cross-modal retrieval algorithm based on text concept expansion, comprising the following steps:

[0039] S1 preprocesses the video to obtain the video embedding feature representation;

[0040] The specific way is:

[0041] S11 extracts key frames from the video according to the preset frame rate to obtain video frames;

[0042] S12 utilizes the residual (ResNet) network to extract the feature of the described video ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of cross-modal retrieval, in particular to a cross-modal retrieval algorithm based on text concept extension, which comprises the following steps: preprocessing a video to obtain video embedded feature representation; preprocessing the text to obtain text embedding feature representation; calculating the similarity between the video embedding feature representation and the text embedding feature representation to obtain the hidden space similarity; deepening the video through an encoder to obtain a video concept; analyzing the text by using grammar analysis to obtain an initial concept; expanding the initial concept through a text concept expansion mechanism to obtain a text concept; calculating the similarity between the video concept and the text concept to obtain the concept space similarity; performing weighted calculation on the hidden space similarity and the concept space similarity to obtain a fusion similarity model; and based on the retrieval instruction, retrieval sorting is performed by using the fusion similarity model, so that the problem that the retrieval performance is reduced due to imbalance of the video information amount and the text information amount is solved.

Description

technical field [0001] The invention relates to the technical field of cross-modal retrieval, in particular to a cross-modal retrieval algorithm based on text concept expansion. Background technique [0002] Cross-modal retrieval needs to use given modal data to find semantically related data from massive information. Therefore, the main problem of cross-modal technology is how to align the semantics of different modal data. [0003] Existing video text retrieval algorithms find a common embedding space for samples of different modalities, and realize the relationship measurement between heterogeneous modalities in this shared space. The advantage of this type of method lies in the diversity of video and text encoders, which can obtain features with strong representation of different modalities, thereby achieving more accurate retrieval. [0004] The above methods use conceptual and deep features as bridges, respectively, but lack in-depth exploration of video text retrieva...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/383G06F16/36G06N5/02G06K9/62G06F16/583G06F40/30
CPCG06F16/383G06F16/367G06N5/02G06F16/5846G06F40/30G06F18/22G06F18/25
Inventor 王树徽方晟
Owner 镇江智栎高科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap