Robot Plume Tracking Method Based on Reinforcement Learning in Continuous State Behavior Domain

Abstract

The invention proposes a robot plume tracking method based on continuous state behavior domain reinforcement learning, which belongs to the field of underwater robot path planning. This method trains the path planning for the underwater robot to search for plume hydrothermal vents; the robot generates a state vector to input the current decision-making neural network in each plume tracking, and the neural network outputs the forward direction of the robot at this moment, and the robot moves at a constant speed. After running for a period of time, update the state vector at the new moment and judge whether the single plume tracking meets the termination condition: when the termination condition is met, the single plume tracking ends, and the robot regenerates a new initial position; if not Satisfied, the robot will continue to move forward at the next moment; in this process, the reinforcement learning algorithm is used to update the decision-making neural network at each moment until the algorithm converges. The invention has fast learning speed and good convergence, can improve the flexibility of the robot to track the plume flow hydrothermal nozzle, and reduce the search cost.

Description

technical field [0001] The invention belongs to the field of underwater robot path planning, in particular to a robot plume tracking method based on continuous state behavior domain reinforcement learning. Background technique [0002] Deep-sea hydrothermal activities and their life phenomena are one of the major discoveries in marine science in the 20th century. Since deep-sea hydrothermal vents are closely related to seafloor spreading and polymetallic sulfide mineralization, and involve cutting-edge scientific issues such as the evolution of biological communities in hydrothermal environments, and the impact of hydrothermal activities on global climate change, etc. The study of deep-sea hydrothermal fluids has become a hot topic in ocean research. [0003] In order to further study deep-sea hydrothermal fluids, it is necessary to explore the location of unknown hydrothermal vents in the deep sea. Researchers have found that deep-sea hydrothermal vents will emit hydrothe...

AI Technical Summary

Problems solved by technology

First of all, there are few judgments, only to make corresponding actions based on whether the concentration is detected and the time when the signal is lost, without using other available information, such as the size and direction of the flow field, the position of the loss, and its own actions when it is lost, etc., which is not considered comprehensive enough

Secondly, the action of the robot in the plume flow is relatively simple. The simple direction of the upstream flow or the direction of a fixed angle deflection with the upstream flow field works well in the narrow, long and straight plume flow, but it is difficult to deal with the complex plume flow in the deep sea.

Finally, the hydrothermal recovery mechanism is a fixed set of search actions. The search range is manually set in advance and cannot adapt to the current environment, resulting in high costs.

Images