Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-language model training method and device, electronic equipment and readable storage medium

A training method and multilingual technology, applied in the field of information processing, can solve problems such as the inability to learn semantic alignment information in different languages, and the inability of multilingual models to accurately realize information interaction in different languages, so as to achieve the goal of strengthening learning ability and improving accuracy Effect

Active Publication Date: 2021-03-19
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF8 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, no matter whether the existing multilingual models use bilingual corpus or monolingual corpus for pre-training, they cannot learn the semantic alignment information between different languages, resulting in the inability of the multilingual model to accurately realize the information interaction between different languages.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-language model training method and device, electronic equipment and readable storage medium
  • Multi-language model training method and device, electronic equipment and readable storage medium
  • Multi-language model training method and device, electronic equipment and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0017] figure 1 is a schematic diagram according to the first embodiment of the present application. Such as figure 1 As shown in , the training method of the multilingual model in this embodiment may specifically include the following steps:

[0018] S101. Obtain training corpus, which includes multiple pieces of bilingual corpus and multiple pieces of monolingua...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multi-language model training method and device, electronic equipment and a readable storage medium, and relates to the technical field of deep learning and natural languageprocessing. According to the technical scheme of the invention, the method comprises the steps: obtaining a training corpus, wherein the training corpus comprises a plurality of bilingual corpora anda plurality of monolingual corpora; training a first training task on the multi-language model by using the plurality of bilingual corpora; performing training of a second training task on the multi-language model by using the plurality of monolingual corpora; and completing the training of the multi-language model under the condition of determining that the loss functions of the first training task and the second training task are converged. Semantic interaction between different languages can be realized by the multi-language model, so that the accuracy of the multi-language model in learning semantic representation of the multi-language corpus is improved.

Description

technical field [0001] The present application relates to the technical field of information processing, in particular to a training method, device, electronic equipment and readable storage medium of a multilingual model in the technical field of deep learning and natural language processing. Background technique [0002] Natural Language Processing (NLP) is a very important subfield of Artificial Intelligence (AI). Most of the existing learning paradigms of NLP tasks adopt the method of pre-training and fine-tuning. Firstly, the pre-training task is used for preliminary modeling in the unsupervised corpus, and then the task data is used for fine-tuning on downstream tasks. And the existing experience shows that the pre-training model can play a role in constraining the regularization of model parameters, which can greatly improve the performance of downstream tasks. Based on the above, and with the continuous development of globalization, the exchange of information betw...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06N20/00
CPCG06F40/30G06N20/00G06F40/58G06N3/08G06N3/045Y02D10/00
Inventor 欧阳轩王硕寰庞超孙宇田浩吴华王海峰
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com