Multi-agent reinforcement learning scheduling method and system, and electronic device

Abstract

The invention relates to a multi-agent reinforcement learning scheduling method, a multi-agent reinforcement learning scheduling system and electronic equipment. The method comprises the following steps: step a, collecting server parameters of a network data center and virtual machine load information running on each server; b, establishing a virtual simulation environment by using the parametersof the server and the load information of the virtual machine, and establishing a deep reinforcement learning model of multiple agents; c, performing offline training and learning by utilizing the deep reinforcement learning model of the multiple agents and a simulation environment, and training an agent model for each server; and d, deploying the agent model to a real service node, and schedulingaccording to the load condition of each service node. The service running on the server is virtualized through the virtualization technology, load balancing is carried out in a virtual machine scheduling mode, resource distribution is more macroscopic, and a strategy that multiple agents generate cooperation in a complex dynamic environment can be achieved.

Description

technical field [0001] The application belongs to the technical field of multi-agent systems, and in particular relates to a multi-agent reinforcement learning scheduling method, system and electronic equipment. Background technique [0002] In the cloud computing environment, it is difficult for the traditional service deployment method to cope with changing access methods. Although the fixed allocation of resources can provide services stably, there is also a large amount of waste of resources. For example, under the same network topology , some servers may often run at full load, while some servers only deploy a few services and still have a lot of unused storage space and computing power. It can be seen that traditional deployment services are difficult to cope with this waste of resources, and it is difficult to implement Efficient scheduling makes it impossible to use resources efficiently. Therefore, a scheduling algorithm that can adapt to the dynamic environment is...

AI Technical Summary

Problems solved by technology

In the field of multi-agent reinforcement learning, if traditional Q-learning, PG (Policy Gradient Method, strategy gradient algorithm) and other reinforcement learning algorithms are used for distributed learning, the expected results will still not be achieved, because each agent in each step All agents try to learn to predict the actions of other agents, and in a dynamic environment, other agents are always changing, so the environment will become unstable, it is difficult to learn knowledge, and it is impossible to achieve optimal resource allocation.

In addition, from the perspective of reinforcement learning methods, most of the current scheduling methods are single-agent reinforcement learning and distributed reinforcement learning. If only one agent is used for centralized training, it will be difficult due to the complex state changes and permutations and combinations under the network topology. A large amount of action space makes the algorithm difficult to train and difficult to converge

The method of using distributed reinforcement learning also faces another problem. The usual distributed reinforcement learning is to speed up the convergence speed through joint training of multiple agents, but in fact the scheduling strategies of these agents are the same, but During the training process, multiple avatars are used to speed up the training, so the final results are homogeneous agents that do not have the ability to cooperate

In the traditional multi-agent method, each agent will predict the decisions of other agents at each step of decision-making, but because the decisions of other agents are also unstable in a dynamic environment, training is very difficult and each agent can do Things are pretty much the same without a collaborative strategy

Images