Distributed stochastic gradient descent method

Abstract

The invention relates to a distributed random gradient descent method. The method comprises the following steps that S1, a parameter server obtains an initial global gradient; s2, based on the initialglobal gradient and an initial task allocation strategy of the working node, the working node calculates and obtains a working node gradient; s3, the parameter server obtains a working node gradient,and calculates and obtains an updated global gradient; s4, the parameter server obtains an optimal gradient and an update task allocation strategy of the working node based on the update global gradient and the block chain technology; s5, the optimal gradient is stored in a parameter cache of the working node; s6, the task allocation strategy is updated to replace the initial task allocation strategy, the initial global gradient is replaced with the optimal gradient, and the steps S2-S6 are repeated until the weight converges. Compared with the prior art, poor model parameters are prevented from being collected, the convergence speed of the model is increased, and the whole-process training time is shortened.

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a distributed stochastic gradient descent method. Background technique [0002] Today, people have realized the great advantages of artificial intelligence technology in many fields. Machine learning is an indispensable work in artificial intelligence. It helps people make judgments and decisions by abstracting and modeling massive data. At the same time, blockchain 3.0 has also been implemented as an application. Its concept has surpassed the fields of digital currency and smart contracts, and can be used as an important supporting technology for the interaction of massive data information. [0003] The rapid growth of massive data is also accompanied by a sharp demand for more complex models (possibly with billions of parameters) to support higher accuracy in orders of magnitude data and solve other intelligent tasks (such as unmanned driving, image processing, etc.) situation...

AI Technical Summary

Problems solved by technology

[0011] The purpose of the present invention is to provide a distributed stochastic gradient descent method to overcome the disadvantages of collecting some poor model parameters and spending a lot of waiting time to recycle and release parameters in the heterogeneous network existing in the above-mentioned prior art

Images