Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A natural language semantic representation method based on an attention mechanism

A technology of natural language and attention, applied in the creation of semantic tools, special data processing applications, instruments, etc., can solve the problem of easily losing the semantic relationship of sentences

Active Publication Date: 2019-04-26
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF5 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, word vectorization often only focuses on the main information of the sentence. For example, only partial semantics are preserved after the convolutional network, and it is easy to lose the semantic relationship of the entire sentence.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A natural language semantic representation method based on an attention mechanism
  • A natural language semantic representation method based on an attention mechanism
  • A natural language semantic representation method based on an attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0039] figure 1 It is a flowchart of an attention mechanism-based natural language semantic representation method in the present invention.

[0040] In this example, if figure 1 Shown, a kind of attention mechanism-based natural language semantic representation method of the present invention comprises the following steps:

[0041] S1, natural language preprocessing

[0042] Randomly download a complete and independent English natural sentence, then remove the special characters in the natural sentence, and then divide the natural sentence into a set of multiple words by spaces S={W 1 ,W 2 ,...,W i ,...,W L}, W i Indicates the i-th word in a natural sentence, and L is the total number of words.

[0043] S2. Obtain the word vector of each word

[0044] Input each word to the Seq2Word model, and output the word vector of each word; where, W i The word vector is expressed as w i ; In this embodiment, the Seq2Word model adopts common word2vec technology, and the dictiona...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a natural language semantic characterization method based on an attention mechanism, which comprises the following steps of: introducing a multi-layer attention mechanism and aposition relation matrix into semantic relation feature extraction so as to obtain more semantic information in language semantic characterization of a natural statement; In this way, the deep learning technology and the attention mechanism are fully combined to share the multi-layer semantic features and the semantic position information features representing the natural language, the semantic features of the language are fused, the algorithm performance is improved, meanwhile, the semantic universality is improved, and the method can be used for various natural language processing tasks.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and more specifically relates to a natural language semantic representation method based on an attention mechanism. Background technique [0002] With the development of artificial intelligence, there is an increasing need for natural language knowledge, semantic derivation, representation and reasoning in life. Through the representation of semantics, it can help to build a computing model to identify the semantics contained in natural language sentences, so that it can understand natural language like a human. [0003] Semantic representation is the fundamental problem of natural language understanding, and it has a wide range of applications in natural language processing, information retrieval, information filtering, information classification, semantic mining and other fields. In the Internet age, faced with massive information resources, semantic analysis oriented to ma...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/332G06F16/36
Inventor 杨波周宇闫新童刘珊曾庆川刘婷婷郑文锋
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com