Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Underwater autonomous robot fixed depth control method based on reinforcement learning

An autonomous robot, fixed depth technology, applied in the direction of height or depth control, adaptive control, general control system, etc., can solve problems such as no application of reinforcement learning control methods

Active Publication Date: 2018-03-02
TSINGHUA UNIV
View PDF4 Cites 40 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] However, these breakthroughs have not been applied to reinforcement learning control methods for AUVs so far.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Underwater autonomous robot fixed depth control method based on reinforcement learning
  • Underwater autonomous robot fixed depth control method based on reinforcement learning
  • Underwater autonomous robot fixed depth control method based on reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0084] A method for controlling the fixed depth of an underwater autonomous robot based on reinforcement learning proposed by the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0085] The present invention proposes a method for controlling a fixed depth of an underwater autonomous robot based on reinforcement learning. The deterministic strategy gradient method is applied to the method for controlling a fixed depth of an AUV, and an artificial neural network with a suitable structure is designed for the AUV control problem. Deterministic policy gradient is a reinforcement learning approach to continuous action spaces that assumes a deterministic policy function and updates the policy along the gradient direction that maximizes the long-term loss function. The present invention focuses on the control problem of AUV fixed depth, and specifically describes how to design a control strategy to dri...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an underwater autonomous robot fixed depth control method based on reinforcement learning and belongs to the underwater robot control field. The method comprises steps that firstly, a Markoff decision process model for underwater autonomous robot fixed depth control is constructed, and expressions of a state variable, a control variable, a transfer model and a one-step lossfunction for underwater autonomous robot fixed depth control are respectively acquired; the decision network and the evaluation network are respectively established; through reinforcement learning, the decision network and the evaluation network can be continuously updated whenever an underwater autonomous robot moves forwards for each step during fixed depth control training till convergence; thefinal decision network for fixed depth control is acquired. The method is advantaged in that fixed depth control on the underwater autonomous robot under the condition of a completely-unknown underwater autonomous robot dynamic model is realized, and the practical value is high.

Description

technical field [0001] The invention belongs to the field of underwater robot control, in particular to a method for controlling a fixed depth of an autonomous underwater robot (AUV) based on reinforcement learning. Background technique [0002] With the development of science and technology and the gradual scarcity of land resources, the demand for ocean exploration research is increasing. As an autonomously controlled intelligent robot, the underwater autonomous robot (AUV) has the advantages of flexibility, safety and reliability, and convenient recycling. It is widely used in various marine exploration scenarios, such as seabed mapping, plume tracking, mineral resource exploration, etc. . Therefore, the control problem of AUV has attracted more and more scholars' interest and attention in the field of control, and related technologies and researches have also made a lot of progress and breakthroughs in recent years. [0003] The traditional AUV control strategy researc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G05D1/04G05B13/04G05B13/02
CPCG05B13/027G05B13/042G05D1/04
Inventor 宋士吉武辉游科友
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com