Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Semantic recognition method and system based on relative position coding

A semantic recognition and relative position technology, applied in the field of semantic recognition, can solve problems such as model output errors, achieve the effect of improving recognition speed, reducing calculation amount, and improving semantic recognition efficiency

Pending Publication Date: 2022-04-29
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For example, when a word appears multiple times in a sentence, if the neural network model cannot distinguish the words that appear before and after and mistakenly think that they are the same, then the output of the model is likely to be wrong

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Semantic recognition method and system based on relative position coding
  • Semantic recognition method and system based on relative position coding
  • Semantic recognition method and system based on relative position coding

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0055] In order to make the above-mentioned features and effects of the present invention more clear and understandable, the following specific examples are given together with the accompanying drawings for detailed description as follows.

[0056] The Transformer uses an external static position encoding, and the dimension of the encoding vector is d model , d model It is determined by the parameters of the decoder model, and it is also the dimension of the embedded encoding (embedding). The specific encoding formula of the position vector PE is as follows:

[0057]

[0058]

[0059] Where pos represents the position, i represents the dimension, where 2i and 2i+1 are used to distinguish whether the dimension is odd or even, the odd and even bits are coded differently, and then the word vector and the position vector are added to get each word final input.

[0060] Relative position encoding:

[0061] 1. The proposal of relative position coding

[0062] According to ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a semantic recognition method and system based on relative position coding, and the method comprises the steps: obtaining a character sequence of a result to be subjected to semantic recognition as training data, splitting the training data, inputting the split training data into a multi-head attention layer in parallel, and calculating the relative position of the split data in the training data through the multi-head attention layer, distributing an attention score for each piece of split data, and generating a relative position code of each piece of split data based on the attention score corresponding to each piece of split data; and the recurrent neural network performs parallel semantic recognition on the split data according to the relative position codes corresponding to the split data, and gathers semantic recognition results corresponding to the split data to obtain semantic recognition results.

Description

technical field [0001] The invention relates to the technical field of semantic recognition, and in particular to a method and system for semantic recognition based on relative position coding. Background technique [0002] For any language, the position and order of words in a sentence are very important. They are not only an integral part of the grammatical structure of a sentence, but also an important concept for expressing semantics. If a word is in a different position or sequence in a sentence, the meaning of the entire sentence may be deviated. For example, when a word appears multiple times in a sentence, if the neural network model cannot distinguish the words that appear before and after and mistakenly think that they are the same, then the output of the model is likely to be wrong. The introduction of word order information helps to understand the original meaning of the sentence, and this word order information is expressed as Position Encode / Embedding (PE). ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06F40/284G06N3/04G06N3/08
CPCG06F40/30G06F40/284G06N3/08G06N3/044
Inventor 杜子东庄毅敏支天
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com