Visual feature segmentation semantic detection method and system in video description

A visual feature and video description technology, applied in the field of deep learning video understanding, can solve problems such as unfavorable security monitoring and short video content review, easy loss of local semantic information, and affect video text description results, so as to improve work efficiency and model performance effect

Active Publication Date: 2021-08-17
DALIAN NATIONALITIES UNIVERSITY
View PDF7 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In the above method of using semantic features and visual features to realize video text description, semantic features only express global semantic information, and it is easy to lose important local semantic information in a certain segment of the video. The error of semantic information will affect the result of video text description, which is not conducive to security. Applications such as surveillance and short video content review

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Visual feature segmentation semantic detection method and system in video description
  • Visual feature segmentation semantic detection method and system in video description
  • Visual feature segmentation semantic detection method and system in video description

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0036] This embodiment proposes a semantic detection method for visual feature segmentation in video description, such as Figure 1-3 As shown, the specific implementation steps are as follows:

[0037] Step 1: The original visual feature vector V obtained after convolution processing the video F As input, read the eigenvector, V F The concrete form is V F ={v 1 ,v 2 ,...v Q} The feature vector of size 1*Q.

[0038] Step 2: In the segmentation semantic detection branch, the original visual feature V in step 1 F Evenly divided into p parts, get p visual segmentation features. As shown in formula (1) and formula (2), the visual segmentation feature V is obtained after segmentation F1 ,V F2 ,...,V Fp .

[0039]

[0040] q=Q / p (2)

[0041] Among them, F a is the uniform partition function, Q is the visual feature V F Dimensions, divide it evenly into p parts, and obtain the visual segmentation feature V Fi The dimensions of all are q, and the specific forms of vi...

Embodiment 2

[0108] Security monitoring outdoor scene

[0109] Apply this example to the outdoor scene of security monitoring to obtain the video semantic features with strong expressive ability, so as to obtain the text description. This text information can effectively prevent the occurrence of outdoor dangerous accidents, and can improve the efficiency of checking surveillance video. Figure 5 shown.

Embodiment 3

[0111] Censorship of short video content

[0112] Apply this example to the short video content review system to obtain video semantic features with strong expressive ability, so as to obtain text description. This text information can effectively prevent negative energy content such as illegal and illegal content in the short video, and is conducive to the construction of a good network environment. The review of short video content is as follows: Figure 6 shown.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a visual feature segmentation semantic detection method and system in video description. Segmenting the visual feature into a plurality of visual segmentation features representing local information; local semantic information is extracted through a multi-layer perceptron, and semantic information with global and local double expressions is obtained after global semantic features are fused; thus, the representation capability of semantic features is enhanced; the obtained semantic features are applied to video description tasks, the precision of a video description model is improved, an accurate video text description result is obtained; therefore, the method can be well applied to the fields of security monitoring, short video content review and the like.

Description

technical field [0001] The invention relates to the technical field of deep learning video understanding, in particular to a semantic detection method and system for visual feature segmentation in video description. Background technique [0002] With the rapid development of information technology, security monitoring equipment is used more and more widely, and with the emergence of a large number of short video platforms, monitoring and automatic review of short video content has become one of the current research hotspots. At present, the censorship of video content mainly relies on manual means, and the computer automatic censorship technology is not mature enough, which cannot fully realize and understand the video content. [0003] Existing video description algorithms increasingly use video semantic features as auxiliary features, and use them together with visual information as encoding features to output corresponding text descriptions in long short-term memory netwo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/06G06N3/08
CPCG06N3/061G06N3/08G06V20/41G06V20/46G06V20/49G06N3/045G06F18/253
Inventor 杨大伟高航毛琳张汝波
Owner DALIAN NATIONALITIES UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap