Network autonomous intelligent management and control method based on deep reinforcement learning

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a network autonomous intelligent management and control method based on deep reinforcement learning. The method comprises the steps of firstly constructing a network topology, then introducing a CNN, an LSTM layer and a delay updating strategy to construct a routing decision-making model based on a DDPG reinforcement learning algorithm, and finally performing iterative training on the routing decision-making model based on deep reinforcement learning. In each iterative training, an intelligent agent obtains an output action, namely a group of link weights, according to a measured network state and a neural network, and calculates a service route by using a shortest path algorithm according to the link weights. According to a routing calculation result, the intelligent agent issues a flow table, and acquires end-to-end time delay and a packet loss probability of the service to calculate a reward value of the iteration. The algorithm has good convergence, and can effectively reduce the end-to-end delay and packet loss rate of the service.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a network autonomous intelligent management and control method based on deep reinforcement learning. Background technique [0002] In recent years, with the expansion of network scale and the increase of application types, formulating intelligent routing strategies for services is an important part of realizing business service quality assurance and independent intelligent network management and control. The emergence of software-defined network (Software Defined Network, SDN) has brought new ideas to the deployment of network autonomous intelligent routing. Different from the tightly coupled vertical structure of traditional networks, SDN separates the data plane and the control plane. The data plane is implemented by SDN switches that support the OpenFlow protocol, and the control plane is implemented by software to provide network programmability. ...

AI Technical Summary

Problems solved by technology

However, using the traditional Q learning algorithm in the SDN network may require a huge storage space to maintain the Q table, and the query of the Q table will also bring additional time overhead

The Deep Q Network (DQN) method can combine the perception ability of deep learning and the decision-making ability of reinforcement learning to optimize the routing process. However, it is limited by the discrete state and action space and is not suitable for dynamic SDN network systems.

Policy-based reinforcement learning methods, such as deterministic policy gradient (Deterministic Policy Gradient, DPG), can be used to deal with continuous action spaces, but they use linear functions as policy functions, and there is an overfitting problem caused by the correlation of training data

Images