Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A Transfer Learning Method Based on Machine Reading to Sequence Model

A transfer learning and model technology, applied in the field of transfer learning based on machine reading to sequence models, can solve the problem of loss of pre-training model information, etc., and achieve the effect of simple and intuitive models, improved quality, and in-depth content

Active Publication Date: 2020-05-29
ZHEJIANG UNIV
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the existing transfer learning methods for natural language processing rarely transfer multi-layer neural networks to other tasks, and only transferring the coding layer will lose a lot of information of the original pre-trained model.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Transfer Learning Method Based on Machine Reading to Sequence Model
  • A Transfer Learning Method Based on Machine Reading to Sequence Model
  • A Transfer Learning Method Based on Machine Reading to Sequence Model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] In order to make the object, technical solution and beneficial technical effect of the present invention clearer, the technical content and specific implementation methods of the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementations described in this specification are only for explaining the present invention, not for limiting the present invention.

[0038] Such as figure 1 As shown, a transfer learning method based on machine reading to sequence model, including the following steps:

[0039] S01, pre-training a machine reading model.

[0040] We use the Stanford Question Answering Dataset SQuAD, a large-scale, high-quality corpus, as the training set. Our task is to predict an answer given an article and a question, which is a continuous field in the article.

[0041] The structure of the machine reading model see figure 2, we use the existing word vector...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a transfer learning method based on machine reading to sequence model, comprising the following steps: (1) pre-training a machine reading model, the machine reading model includes a coding layer and a model layer based on a recurrent neural network; (2) ) set up a sequence model, the sequence model includes an encoder and a decoder based on a recurrent neural network; (3) extract the parameters of the encoding layer and the model layer in the trained machine reading model, and migrate to the sequence model to be trained, As part of the initialization parameters when training the sequence model; (4) train the sequence model until the model converges; (5) use the trained sequence model for text sequence prediction tasks. By using the present invention, the information contained in the text can be more deeply excavated, and the quality of the generated text sequence can be improved.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a transfer learning method based on machine reading to sequence model. Background technique [0002] Machine reading is one of the hottest and thorniest problems in natural language processing, which requires models to understand natural language and be able to apply existing knowledge. The most popular task at present is generally given an article and a question, and we need to find the answer from the article according to the question. With the release of several high-quality datasets in recent years, the performance of neural network-based models on machine reading is getting better and better, even surpassing humans on some datasets. An efficient machine reading model can be widely used in many fields based on semantic understanding, such as dialogue robots, question answering systems and search engines. [0003] The sequence model with atten...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/30G06F16/332G06N3/04G06N3/08
CPCG06N3/08G06F40/30G06N3/045
Inventor 潘博远蔡登李昊陈哲乾赵洲何晓飞
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products