A multimodal surgical trajectory fast segmentation method based on unsupervised deep learning

Abstract

The invention discloses a multimodal surgical trajectory fast segmentation method based on unsupervised deep learning, belonging to the field of robot assisted minimally invasive surgery. Firstly, therobot system collects kinematics data and video data for a robot-assisted minimally invasive surgery process. Then wavelet transform is used to smooth and filter the short trajectory noise and data jitter in kinematics data. At the same time, a DCED-Net network structure is used to extract features from video data. The kinematics data after smoothing filtering and the video data image after feature extraction are sent to the improved TSC model for clustering, and the pre-segmentation results of n surgical demonstration trajectories are obtained. Finally, the PMDD merging algorithm is used tosimilarly merge the pre-segmentation results of each trajectory, and the merging result is the final trajectory segmentation result. The invention provides an optimization scheme and unsupervised deeplearning for over-segmentation and other problems, accelerates the extraction speed of video features, improves the quality of features and makes the clustering result more accurate.

Description

technical field [0001] The invention belongs to the field of robot-assisted minimally invasive surgery (RMIS), and relates to image feature extraction, deep learning clustering, similarity evaluation, etc., and is specifically a method for fast segmentation of multimodal surgical trajectories based on unsupervised deep learning. Background technique [0002] During robot-assisted minimally invasive surgery (RMIS), the surgical trajectory is recorded by a series of robot kinematics data and video data. By segmenting these surgical trajectories, the surgical process is decomposed into several low-complexity sub-trajectories (sub-actions), which can be used for doctor skill evaluation and demonstration learning. More importantly, by learning these sub-trajectories, the robot can realize the autonomous operation of simple tasks, thereby advancing the automation of robotic surgery. However, due to the complexity of the surgical environment and the differences in the skill levels...

AI Technical Summary

Problems solved by technology

[0007] However, there are still many defects in the existing unsupervised trajectory segmentation methods. First, the slowness of video feature extraction is the main problem affecting medical trajectory segmentation. For example, TSC-VGG, whose video feature extraction time accounts for more than 95% of the total segmentation time , so that the efficiency of unsupervised methods is greatly reduced; secondly, the extracted video features are not significant

The quality of the video features extracted by the existing method is poor, and it may even have a negative effect in the trajectory segmentation, resulting in poor segmentation stability; finally, due to the characteristics of the unsupervised trajectory segmentation method itself, the result will have the problem of over-segmentation, that is, The split segment of the same atomic operation is divided into multiple segments, and some split segment "fragments" appear

Images