Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Underwater vehicle docking control method based on reinforcement learning

An underwater vehicle and reinforcement learning technology, applied in the direction of height or depth control, sustainable transportation, etc., can solve problems such as docking control of underwater vehicles, so as to promote the completion of docking tasks, promote high robustness, and improve autonomous vehicles. The effect of learning ability

Active Publication Date: 2022-07-08
SHANDONG UNIV
View PDF13 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of the deficiencies in the prior art, in order to solve the problem of underwater vehicle docking control, the present invention is based on the PPO algorithm framework in deep reinforcement learning, and proposes an underwater vehicle docking control method based on adaptive reliable boundary rollback clipping reinforcement learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Underwater vehicle docking control method based on reinforcement learning
  • Underwater vehicle docking control method based on reinforcement learning
  • Underwater vehicle docking control method based on reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0141] An underwater vehicle docking control method based on reinforcement learning, comprising the following steps:

[0142] Step 1. Define the task environment and model

[0143] 1-1. Build the task environment where the underwater vehicle is located and the dynamic model of the underwater vehicle;

[0144] The task environment consists of Coordinate system, 3D area with 3D map size set, 3D cone docking station area;

[0145] The underwater vehicle includes three actuators, namely the stern thruster, the stern horizontal rudder and the stern vertical rudder;

[0146] Through the derivation based on the Newton-Eulerian equation of motion in the simulation, a streamlined underwater vehicle with a length of 2.38 meters, a diameter of 0.32 meters, and a weight of 167 kilograms is modeled with six degrees of freedom, including, in The coordinates of the underwater vehicle in the coordinate system and attitude angle To describe, use the center of gravity as the origin, wh...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to an underwater vehicle docking control method based on reinforcement learning, belongs to the technical field of ocean control experiments, and introduces a reliable boundary updated by new and old strategies based on a PPO algorithm framework in deep reinforcement learning to improve the stability of agent learning. Meanwhile, a self-adaptive rollback cutting mechanism is adopted, the rollback strength is adjusted in a self-adaptive mode according to the situation that task successful completion experience is collected, and therefore the upper limit and the lower limit of new and old strategy updating are adjusted, and an intelligent agent is encouraged to explore in the initial training stage and converge stably in the later training stage. In the aspect of simulation training, a docking training environment considering ocean current and ocean wave interference is constructed, the training environment is used for intelligent body learning, and the anti-interference capability of the underwater vehicle is greatly improved.

Description

technical field [0001] The invention relates to an underwater vehicle docking control method based on reinforcement learning, and belongs to the technical field of marine control experiments. Background technique [0002] As a special marine survey equipment, underwater vehicles have been widely used in many marine engineering fields such as seabed topographic mapping, marine resource exploration, shipwreck monument investigation, oil and gas pipeline maintenance, life science monitoring, etc. indispensable means. However, due to the need to ensure the flexibility of the underwater vehicle itself and to carry corresponding equipment, the limited energy carried by the underwater vehicle limits its long-term cruise capability, and it is inevitable to replenish the energy regularly. In order to prevent underwater vehicles from relying on surface ships for energy supplementation, so that they have fully automatic long-term operation capabilities, such as the Chinese patent docu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G05D1/06
CPCG05D1/0692Y02T90/00
Inventor 李沂滨张天泽缪旭弘魏征尤岳周广礼贾磊庄英豪宋艳
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products