Model parameter fusion method and device

A technology of model parameters and fusion devices, which is applied in the field of machine learning and can solve the problems of high performance requirements of parameter servers and large data transmission volumes

Active Publication Date: 2020-04-28
HUAWEI TECH CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Embodiments of the present invention provide a model parameter fusion method and device, which are used to solve the problems of high performance requirements for parameter servers and large data transmission volumes in model parameter fusion

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model parameter fusion method and device
  • Model parameter fusion method and device
  • Model parameter fusion method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0131] The architecture of the machine learning system applied in the embodiments of the present invention is as follows: figure 1 As shown, the system architecture diagram includes a data storage device 101 , a model parameter training platform 102 and a model parameter storage device 103 .

[0132] Wherein, the data storage device 101 can be a data storage server 101, and the data storage server 101 can be used to store raw data used for model parameter training. capacity. The original data can be language data, image data, and video data, etc., and the original data is composed of multiple data sets, and each data set is composed of multiple type subsets, and each type subset has a The data labels of the type subsets included in the same data set are the same. For example, the data set can contain multiple images of people with human labels, or it can contain multiple images of animals with animal labels. , or other categories of images, etc.

[0133] The model parameter...

Embodiment 2

[0136] figure 2 It is a flow chart of a model parameter fusion method provided by an embodiment of the present invention, the method is applied to a machine learning system, and the machine learning system includes M nodes, and the method includes the following steps.

[0137] Step 201: The node used for model parameter fusion acquires a data subset in the data set.

[0138] Among them, the data set refers to a data set used for iterative calculation of model parameters. The data set can be language data, image data, and video data, etc., and the data set is composed of multiple types of subsets, each type of sub-set Sets have data labels that represent categories, and the labels for the subset of types included in the same dataset are the same.

[0139] In addition, the data set can be stored in advance on storage devices such as hard disks and disks, or on a data storage server. When a node obtains a subset of data from the data set, the storage device can be directly conn...

Embodiment 3

[0182] Figure 5 A schematic structural diagram of a model parameter fusion device provided by an embodiment of the present invention is applied to a machine learning system, and the machine learning system includes M nodes, such as Figure 5 As shown, the device includes:

[0183] The division unit 301 is configured to divide its own model parameters into N blocks; wherein, the N is the number of model parameter fusion devices participating in fusion among the M model parameter fusion devices, and the N blocks divided by the model parameters The i-th block in is the i-th block model parameter, 1 ≤ said i ≤ said N ≤ said M; the i-th block model parameter here refers to the corresponding i-th model parameter fusion device in the divided N block model parameters For a block of model parameters, the i-th model parameter fusion device is responsible for performing subsequent fusion operations on the i-th block of model parameters.

[0184] The first receiving unit 302 is configu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Embodiments of the present invention provide a model parameter fusion method and device, which relate to the field of machine learning and are used to solve the problems of large data transmission volume and dynamic adjustment of computing resources in model parameter fusion. The method includes: the i-th node divides the model parameters of the i-th node into N blocks; wherein, the i-th node is any one of the N nodes participating in the fusion, and 1≤the i≤the N≤ The M; the i-th node receives the respective i-th block model parameters sent by other nodes except the i-th node in the N nodes; the i-th node sends the i-th node's The i-th block model parameters and the respective i-th block model parameters sent by other nodes are fused to obtain the total model parameters of the i-th block; the i-th node distributes the total model parameters of the i-th block to the N nodes other nodes in the .

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a model parameter fusion method and device. Background technique [0002] Model parameters refer to the parameters describing the model composed of multiple constraint parameters. Through the model parameters, data with common characteristics can be filtered out. For example, when the model parameters are image-type model parameters, different model parameters can be selected from many Filter out image data with people, animals, or human faces from the image data. With the rapid growth of data volume and data types, more and more model parameters are used for data screening, and these model parameters are obtained through multiple calculations and fusion of a large number of data with common characteristics. [0003] At present, the model parameter fusion is to divide the data into multiple data subsets, assign them to different nodes, and use the data iteration calculation metho...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N20/00
CPCG06F17/18G06N20/00G06F16/00H04Q3/5455G06F18/25
Inventor 徐君邵云峰杨肖
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap