Fixed-wing unmanned aerial vehicle cluster control collision avoidance method and device based on deep reinforcement learning

Abstract

The invention discloses a fixed-wing unmanned aerial vehicle cluster control collision avoidance method and device based on deep reinforcement learning. The method comprises the steps of S1, establishing an unmanned aerial vehicle kinematic model and D3QN, updating network parameters by using historical interaction data in the interaction process of each wing plane and the environment; training toform a D3QN model, constructing a joint state between the wing planes and a lead plane according to the environment state in the interaction process, performing situation assessment construction to obtain a local map, and inputting the local map into the D3QN model to obtain control instruction output of each wing plane; S2, obtaining state information in real time by each wing plane to form a joint state between the current wing plane and the lead plane, carrying out situation assessment in real time to construct a local map, and inputting the joint state constructed in real time and the local map into the D3QN network model to obtain a control instruction of each wing plane. The method has the advantages that the implementation method is simple, the expandability is good, cluster control over the fixed-wing unmanned aerial vehicles can be achieved, and meanwhile, collision is avoided.

Description

technical field [0001] The invention relates to the technical field of fixed-wing unmanned aerial vehicle cluster control, in particular to a fixed-wing unmanned aerial vehicle cluster control collision avoidance method and device based on deep reinforcement learning. Background technique [0002] With the continuous development of UAV system technology, UAVs have been widely used in various military operations and civilian tasks such as disaster search and rescue, geographic surveying and mapping, and military reconnaissance. In recent years, the application style of drones has gradually changed from single platform to multi-platform, and is developing in the direction of clustering. Although UAVs have made great progress in terms of operational autonomy in recent years, it is still a big challenge to efficiently and conveniently control UAV swarms in dynamic environments. [0003] At present, UAV swarm control methods can be divided into two categories: rule-based methods...

AI Technical Summary

Problems solved by technology

However, in the prior art, UAV swarm control solutions based on reinforcement learning are usually mainly aimed at rotor UAVs. Unlike rotor UAVs, due to the non-holonomic constraints of the flight dynamics of fixed-wing UAVs, fixed The swarm control of wing UAVs is more complicated, and the control strategy suitable for rotorcraft cannot be directly applied to the swarm control of fixed-wing UAVs.

[0004] Some practitioners proposed to use deep reinforcement learning method to solve the fixed-wing UAV cluster control problem, but this research is still in its infancy, and it is realized by simplifying the problem. Generally, it is assumed that UAVs fly at different altitudes. The collision problem between UAVs is not considered, but in some practical application scenarios, UAV formations need to fly at the same altitude to perform tasks, and avoiding collisions between UAVs is a problem that must be considered. The control method is used to realize the cluster control of fixed-wing UAVs, and there will be a greater risk of collision between UAVs

Images