Ship intelligent collision avoidance method based on reinforcement learning

Abstract

The invention discloses a ship intelligent collision avoidance method based on reinforcement learning. The ship intelligent collision avoidance method based on reinforcement learning comprises the steps of firstly, obtaining static data and dynamic data of two ships; secondly, checking the validity of the data and judging whether a collision avoidance program needs to be started or not; calculating a relevant collision avoidance parameter and judging whether a dangerous situation can be caused or not; if the collision danger cannot be generated, enabling the ships to advance by keeping speedsand directions according to a collision avoidance rule; if the collision danger can be generated, using a reinforcement learning method to learn a collision avoidance strategy, using input data as thecalculated parameter to perform training, outputting a strategy generated after training and obtaining a rudder angle for which the ship needs to steer; thirdly, executing the strategy, dynamically updating the dynamic data of the two ships in the step 1 and returning a bonus value; and fourthly, determining a reversion time of course according to the collision avoidance rule after the executionof the strategy is finished, and then enabling the ship to reverse the course. According to the ship intelligent collision avoidance method based on reinforcement learning, autonomic learning and improvement of ship collision avoidance are realized, and an unfavorable situation caused by sailors and so on absolutely relying on experiences is avoided.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence and relates to an intelligent collision avoidance method for ships, in particular to an intelligent collision avoidance method for ships based on reinforcement learning. Background technique [0002] In the process of navigation, ship collision avoidance is a problem that cannot be ignored. There are many different solutions to this problem, using intelligent decision-making for ship collision avoidance based on AIS, using intelligent algorithms for ship collision avoidance based on evolutionary genetic algorithms, and based on Bayesian networks These algorithms have a certain ability to solve the problem of ship collision avoidance, but they also have their limitations. They cannot self-learn and improve the collision avoidance strategy. [0003] At present, the problem of ship avoidance in open waters involves multiple ships. The existing methods of ship collision avoidance in op...

AI Technical Summary

Problems solved by technology

[0002] In the process of navigation, ship collision avoidance is a problem that cannot be ignored. There are many different solutions to this problem, using intelligent decision-making for ship collision avoidance based on AIS, using intelligent algorithms for ship collision avoidance based on evolutionary genetic algorithms, and based on Bayesian networks The ship collision avoidance algorithm, etc. These algorithms have a certain ability to solve the ship collision avoidance problem, but they also have their limitations, they cannot self-learn and improve the collision avoidance strategy

[0003] At present, the problem of ship avoidance in open waters involves multiple ships. The existing methods of ship collision avoidance in open waters are mainly based on the "International Regulations for Preventing Collisions at Sea". , in the actual process of ship avoidance, the common practice of seafarers, the actual experience of the pilot, etc. will have a significant impact on the specific decision-making scheme and the effect of ship collision avoidance

[0004] In the actual situation, ship collision avoidance is mainly controlled by people, and it is very dependent on the usual practice of the seafarers and the actual experience of the driver, so there is a lot of instability

Images