Parameter synchronous multicast method for distributed machine learning

Abstract

The invention discloses a parameter synchronous multicast method for distributed machine learning. The method comprises the following steps that a multicast tree corresponding to a sender node is generated based on a connection relationship of all working nodes in a data center network; the sender node obtains the target sending rate, and sends data through the multicast tree based on the target sending rate; a switch node receives the data and forwards the data to a receiver node; the receiver node receives the data, updates receiver multiplier parameters, and sends the receiver multiplier parameters to the switch node; the receiver node receives the receiver multiplier parameter, updates a receiver synchronization parameter and a sender synchronization parameter, and sends the receiver synchronization parameter to the sender node, so that the sender node updates a target sending rate based on the receiver synchronization parameter, and sends the sender synchronization parameter to the receiver node, so that the receiver node updates the receiver multiplier parameter based on the sender synchronization parameter. The method can guarantee the network communication efficiency of the data center network.

Description

technical field [0001] The invention relates to the technical field of distributed machine learning, in particular to a parameter synchronization multicast method oriented to distributed machine learning. Background technique [0002] As the complexity of machine learning tasks increases, the scale of data and deep learning models becomes increasingly large. The single-computer computing method can no longer meet the computing power and storage space required by large-scale deep learning models. In order to improve the training efficiency of deep learning models, researchers use distributed machine learning to perform large-scale training tasks in order to efficiently train deep learning models. Mass data is the foundation of machine learning, and computing power is a necessary condition for machine learning. A data center network with powerful computing power is just in line with the computing power needs of deep learning models for distributed training. Since the existi...

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present invention provides a parameter synchronous multicast method oriented to distributed machine learning, so as to solve the problem that there are a large number of redundant transmissions of repeated data streams in the existing process of parameter synchronous multicast oriented to distributed machine learning, which leads to a decrease in communication efficiency. question

Images