A Reinforcement Learning Method for Unmanned Aerial Vehicle Swarms Cooperatively Searching for Multiple Dynamic Targets in Uncharted Seas

Abstract

The invention discloses a reinforcement learning method for collaboratively searching multiple dynamic targets in an unknown sea area by a swarm of unmanned aerial vehicles, comprising the following steps: S1: using a grid method to divide the search sea area; Concentration to establish territorial awareness information map S2: Design the Q value table according to the UAV state information and decision-making u(k); S3: According to the Q value of the current state of the UAV swarm, the Boltzmann distribution mechanism is used to select the UAV's flight route and execute it; S4: Use the search efficiency function to design a reward and punishment function for evaluating the flight state of the UAV, and update the Q value of the new state of the UAV swarm according to the reward and punishment function; S5: Update the new state of the UAV swarm to the current state, Continue to make flight route decisions and finally complete the learning of the entire Q-value table. The UAV group makes decisions based on the trained Q-table to complete the search task.

Description

technical field [0001] The invention relates to the technical field of unmanned aerial vehicle control, in particular to a method for cooperatively searching multi-dynamic targets in an unknown sea area by a group of unmanned aerial vehicles through reinforcement learning. Background technique [0002] With the rapid development of technologies such as sensors, wireless communication, and intelligent control, the functions of unmanned swarm systems are increasingly enhanced, and their application fields are expanding. Due to its scalability, strong collaboration and low loss, the unmanned swarm system has attracted more and more attention from academia, industry and national defense in its collaborative theory and application research, and the multi-UAV cooperative search system can effectively improve the search Efficiency, especially for the search of dynamic targets in complex sea conditions such as uncertainty and strong interference, has great advantages. Therefore, mul...

AI Technical Summary

Problems solved by technology

[0003] The traditional search method is to use coverage search, such as back-type search, traversal search, etc. This search method generally maximizes the coverage of the task area to find as many targets as possible. In recent years, a search graph model has been established in combination with the target existence probability , using distributed model predictive control to solve, effectively reducing the solution scale of the search decision problem, but only limited to the search of static targets

For dynamic targets, the Bayesian method is used to calculate the average detection time and average detection probability, but it is only applicable to the search for a single target at sea, and cannot meet the needs of multi-target search

Images