Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Attention-based joint source channel method for text

A technology of attention and text, applied in the field of wireless communication, can solve problems such as inability to adapt to complex network environments

Active Publication Date: 2021-08-24
CENT SOUTH UNIV
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the technical problem that the current source coding and channel coding are designed separately and cannot adapt to the existing complex network environment, the present invention provides an attention-based joint source-channel method for text

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Attention-based joint source channel method for text
  • Attention-based joint source channel method for text
  • Attention-based joint source channel method for text

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0047] This embodiment implements JSCC for text data with a fixed code length based on deep learning and using GRU. This embodiment proposes to include two parts:

[0048] 1) In this embodiment, GAN is used to train the unidirectional JSCC model, and pointwise mutual information (Pointwise Mutual Information, PMI) is added in the process of beam search.

[0049] 2) In order to overcome the deficiency of unidirectional JSCC, this embodiment adopts Synchronous Bidirectional Attention (SBAtt) and synchronous bidirectional beam search to interactively use past and future information to decode text.

[0050] 1) GAN training one-way neural network

[0051] see figure 1 , the SeqGAN model consists of three parts: generator G, discriminator D and sentence-level word error rate (Word Error Rate, WER). The decoding end of the generator is based on the one-way GAN neural network. The whole generator can be regarded as a JSCC framework, the purpose is to decode the input sou...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an attention-based joint source channel method for a text, which uses a generator comprising an encoder, a channel and a decoder to carry out joint source channel coding on the text, and uses a GRU to replace an LSTM, so that the calculation speed is higher. Meanwhile, an attention mechanism is added, so that the decoding efficiency and accuracy are improved. By using the GAN framework, the parameter update of the generated model is not directly from an original data sample but from the back propagation of a discrimination model. According to the method, a one-way text decoding framework based on GRU is used as a generation model, a CNN is used as a discrimination model, and samples generated by a generator are discriminated. Point mutual information is added in beam search, and words are decoded more accurately together with the maximum likelihood probability. At a decoding end, a synchronous bidirectional decoding method is used. Meanwhile, L2R and R2L decoding is carried out, and alignment and information interaction are realized by utilizing SBAtt and bidirectional beam search, that is, past and future information is utilized at the same time, so that the word error rate of decoding is reduced.

Description

technical field [0001] The invention relates to the field of wireless communication, in particular to an attention-based joint source-channel method for text. Background technique [0002] With the development of computer technology, modern communication technology and network information processing technology, text has become the main carrier for users to obtain and disseminate information. The encoding and transmission of text data also faces great challenges. On the one hand, in many application occasions, there will be a problem of needing to transmit massive text data information. In order to transmit a large amount of information in a channel with limited bandwidth, high-efficiency compression coding must be performed before transmission. On the other hand, due to the increasing complexity of wireless channels, how to effectively deal with the impact of channels on text transmission is also an urgent problem to be solved. [0003] Shannon's separation theorem points...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L1/00G06N3/04G06N3/08
CPCH04L1/0009H04L1/0014G06N3/08G06N3/047G06N3/048G06N3/044G06N3/045
Inventor 陈雪晨刘婷
Owner CENT SOUTH UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com