Network optimization method for oblique-tip flexible needle path tracking based on deep reinforcement learning

Abstract

The invention belongs to the field of puncture needle path tracking, and discloses a network optimization method for oblique-tip flexible needle path tracking based on deep reinforcement learning. Themethod comprises the following steps: (1) constructing a simulation environment based on a bicycle model of an oblique-tip flexible needle; (2) initializing the whole oblique-tip flexible needle model, and meanwhile, providing a pre-planned preset tracking path; (3) constructing an action network and a target action network; (4) integrally storing [s(t), a(t), s(t + 1), R(t + 1)] as one sample into an empirical pool replay variable until the replay variable is saturated; and (5) randomly taking out a plurality of samples as training samples, training the action network and the target action network, and ensuring convergence of the two networks at the same time, wherein the trained convergent action network is the obtained action network for oblique-tip flexible needle path tracking. The action network obtained through the optimization method can be used for tracking the complex three-dimensional puncture path of the flexible needle, and compared with a traditional method based on theduty ratio, the method has smaller path tracking error.

Description

technical field [0001] The invention belongs to the field of puncture needle path tracking, and more specifically relates to a network optimization method for path tracking of inclined-tip flexible needles based on deep reinforcement learning. The optimized action network can be used to realize the purpose of tracking the three-dimensional path of flexible needles. Background technique [0002] Puncture surgery is one of the most widely used operations in clinical practice. In puncture surgery, the commonly used puncture needles are mostly rigid needles, but they can only advance in a straight line, and it is difficult to effectively avoid obstacles such as blood vessels. The flexible puncture needle can advance along the curve, thereby avoiding important blood vessels and organs, reducing the damage to the human body during puncture surgery, and reducing the pain of patients. Therefore, flexible puncture needles have received extensive attention in puncture surgery in recen...

AI Technical Summary

Problems solved by technology

[0010] Generally speaking, the flexible needle path tracking methods developed at home and abroad have shortcomings such as limited puncture range or relatively single puncture path form.

Images