Multi-step communication distributed optimization method based on Barzilai-Borwein step length

Abstract

The invention relates to a multi-step communication distributed optimization method based on a BB step length, and belongs to the technical field of large-scale machine learning and information processing. The method comprises the following steps: 1), determining a problem objective function and a strong convex coefficient and a smooth coefficient thereof; 2), constructing a directed strongly-connected unbalanced communication network, and generating a row-column random weight matrix by adopting a uniform weight strategy according to the adjacency matrix of network topology; 3), initializing all variables of the system; 4), calculating a BB step length by utilizing variable information currently stored by the node; 5), enabling the system to calculate the number P of internal circulation times according to the related parameters; and 6), enabling the nodes to receive information of the neighbor nodes and updating the variables in combination with the information stored in the nodes until the variables converge, and obtaining the optimal value of the problem objective function. The method can solve the problem of unconstrained optimization under the directed strongly-connected unbalanced communication network, and the convergence rate and gradient estimation accuracy of the algorithm are improved.

Description

technical field [0001] The invention belongs to the technical field of large-scale machine learning and information processing, and relates to a multi-step communication distributed optimization method based on BB step size. Background technique [0002] With the development of the Internet, computers, and artificial intelligence, data is flooding, and how to effectively process data is becoming an urgent problem to be solved in various industries. Currently, there are two effective data processing methods: centralized optimization algorithms and distributed optimization algorithms. The centralized optimization algorithm mainly relies on the computing power of the central processor to analyze and process the data. When the data is simple and the network scale is small, the centralized optimization algorithm can efficiently process the data. However, since the central processor is limited by the development of physical hardware, its computing power is extremely limited. Th...

AI Technical Summary

Problems solved by technology

In addition, the distributed optimization algorithm hardly uses a multi-step communication mechanism, which also leads to a large gradient estimation error

[0003] To sum up, the problems existing in the existing technology are: 1) the existing centralized algorithm is not capable of processing large-scale network tasks, and there are problems such as poor robustness and single point of failure; 2) the existing distributed optimization algorithm Judging the advantages of the algorithm only by the number of iterations is too one-sided; 3) Most of the existing distributed optimization algorithms use a constant step size, which can only be manually adjusted to ensure the theoretical derivation results; 4) The existing distribution The formula optimization algorithm does not give a solution to the gradient estimation error

Generally speaking, an undirected network is a special case of a directed network, and the distributed algorithm design based on a directed network is more difficult than the distributed algorithm design based on an undirected network.

Images