A Data Center Network Load Balancing Method Based on Deep Reinforcement Learning

Abstract

The invention discloses a data center network load balancing method based on deep reinforcement learning. Step 1: Build a virtual network topology environment; Step 2: Build and initialize the Actor network, Critic network, target Actor network and target Critic network. Step 3: Each Input the flow information in the network into the network constructed in step 2 at intervals, and perform DDPG training on the link weight optimization problem until the ideal FCT value of the network is reached; use the training objective of deep reinforcement learning to maximize the expectation of cumulative rewards, Finally a decision tree is extracted from the DNN. The present invention designs an efficient and lightweight data center load balancing method; the decision tree is lighter and the reasoning time is shorter, so that the controller can notify the terminal host of the updated link weight faster; The policy gradient algorithm is applied to the load balancing strategy of the data center network to balance the traffic load among multiple paths.

Description

technical field [0001] The invention belongs to the technical field of computer networks, and in particular relates to a method for realizing load balancing in a data center network. Background technique [0002] The most common topology for data center networks is the multi-rooted tree topology. This regular topology allows multiple equal-cost paths between end-to-end, thus providing a large amount of bisection bandwidth. When the network load is uneven, some links or paths are congested, while the utilization rate of other links is not high, resulting in reduced network throughput and increased latency. Therefore, designing a reasonable and effective traffic scheduling strategy is critical to improving the performance of throughput-sensitive and delay-sensitive applications. Equal-Cost Multipath Routing (ECMP) is currently the most commonly used load balancing solution in data centers. The switch locally selects the corresponding path for the flow according to the hash r...

AI Technical Summary

Problems solved by technology

But the routing decision made by the agent for each flow inevitably leads to a long decision delay

Because the majority of data center traffic is short flows, most flows end before their decision arrives, and the decision becomes useless

And, for better performance, DRL agents may have to use large deep neural network models with millions or even billions of parameters, which makes the situation worse the longer the decision time becomes

Images