Text processing method and device and computing equipment

A processing method and text processing technology, applied in computing, neural learning methods, electrical digital data processing, etc., can solve the problems of model performance loss, high overhead, long training time, etc., and achieve the effect of model performance improvement

Active Publication Date: 2020-06-12
ALIBABA GRP HLDG LTD
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] For example, for a general model compression algorithm, one method is simply to eliminate the parameters that are not very important in the model, which is easy to cause loss in the performance of the model; the other is to use a small model to learn a large model method to obtain a small model with fewer parameters. On the one hand, this method may lose performance, and on the other hand, it will increase the training cost of the model.
[0006] For the problem of unregistered words, one method ...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text processing method and device and computing equipment
  • Text processing method and device and computing equipment
  • Text processing method and device and computing equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0042] A language model is a machine learning model used to model the probability distribution of a continuous sequence. Using a language model, it is possible to determine which sequence of words is more likely, or, given several words, to predict the next most likely word.

[0043]Language models are widely used in practical application scenarios, such as query error correction, text similarity calculation, speech recognition, tr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a text processing method and device and computing equipment. The method comprises the steps that a stroke vector matrix is generated based on stroke information of a single character in a word, the single character is an ideographic character, and strokes are minimum continuous stroke units of the single character; inputting the stroke vector matrix into a first neural network for processing to obtain word vectors of words; word vectors of all words in the word sequence are obtained and input into a second neural network to be processed, hidden layer representation of the word sequence is obtained, and the hidden layer representation is used for representing the distribution probability of the next word of the word sequence.

Description

technical field [0001] The present invention relates to the field of natural language processing, in particular to a text processing method, device and computing equipment. Background technique [0002] Complex machine learning language models, especially those based on neural networks, have huge model parameters, which makes the model have higher requirements for storage and operating environments; and a large part of the parameters are occupied in the model In terms of word vector representation, the vocabulary of the model can be as high as more than 200,000 words, and the dimension of each word vector can be hundreds of dimensions. At the same time, although there is a very large vocabulary, this vocabulary is generally fixed during model training. In reality, new words are invented every day. These words that are not in the vocabulary (called out-of-the-table Words or unregistered words) are very difficult to deal with models with fixed vocabularies. [0003] For exam...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F40/232G06F40/289G06N3/08G06F16/332
CPCG06N3/08
Inventor 包祖贻徐光伟李辰刘恒友李林琳司罗
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap