A network autonomous intelligent control method based on deep reinforcement learning

Abstract

The invention belongs to the technical field of artificial intelligence, and specifically relates to a network autonomous intelligent management and control method based on deep reinforcement learning. The present invention first constructs a network topology, then introduces CNN, LSTM layer and delay update strategy to construct a routing decision model based on DDPG reinforcement learning algorithm, and finally iteratively trains the routing decision model based on deep reinforcement learning. In each iteration training, the agent obtains the output action according to the measured network state and the neural network, that is, a set of link weights, and uses the shortest path algorithm to calculate the service route according to the link weights. According to the route calculation result, the agent issues the flow table, and obtains the end-to-end delay and packet loss rate of the business to calculate the reward value of this iteration. The algorithm has good convergence and can effectively reduce the end-to-end delay and packet loss rate of the service.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a network autonomous intelligent management and control method based on deep reinforcement learning. Background technique [0002] In recent years, with the expansion of network scale and the increase of application types, formulating intelligent routing strategies for services is an important part of realizing business service quality assurance and independent intelligent network management and control. The emergence of software-defined network (Software Defined Network, SDN) has brought new ideas to the deployment of network autonomous intelligent routing. Different from the tightly coupled vertical structure of traditional networks, SDN separates the data plane and the control plane. The data plane is implemented by SDN switches that support the OpenFlow protocol, and the control plane is implemented by software to provide network programmability. ...

AI Technical Summary

Problems solved by technology

However, using the traditional Q learning algorithm in the SDN network may require a huge storage space to maintain the Q table, and the query of the Q table will also bring additional time overhead

The Deep Q Network (DQN) method can combine the perception ability of deep learning and the decision-making ability of reinforcement learning to optimize the routing process. However, it is limited by the discrete state and action space and is not suitable for dynamic SDN network systems.

Policy-based reinforcement learning methods, such as deterministic policy gradient (Deterministic Policy Gradient, DPG), can be used to deal with continuous action spaces, but they use linear functions as policy functions, and there is an overfitting problem caused by the correlation of training data

Images