Semantic SLAM robustness improvement method based on instance segmentation

Abstract

The invention relates to a semantic SLAM robustness improvement method based on instance segmentation, and the method comprises the steps: firstly carrying out the instance segmentation of a key framethrough an instance segmentation network, and building prior semantic information; calculating a feature point optical flow field to further distinguish the object, identifying a real moving object in the scene, and removing the feature points belonging to the dynamic object; and finally, performing semantic association, and establishing a semantic map without dynamic object interference. Compared with the prior art, the semantic map is established by adopting a method of combining deep learning and optical flow, and the depth map is added on the basis of the color map, so that the system isendowed with the capability of establishing the dense three-dimensional point cloud semantic map. In addition, a Mask-RCNN framework is adopted for real-time semantic segmentation, and object dynamicinformation can be calculated through mutual combination of dynamic feature points estimated by optical flow information and pixel-level semantic information. According to the method, deep learning and optical flow are mutually combined, so that the robustness of the whole system is remarkably improved, and the method can be applied to real-time semantic map construction in a dynamic scene.

Description

technical field [0001] The invention relates to the field of instance segmentation and semantic SLAM technology, in particular to an improved method for dynamic scene semantic SLAM based on deep learning. Background technique [0002] Simultaneous localization and mapping (SLAM) refers to the process of a robot estimating its own pose and constructing an environmental map only through its own sensors in an unfamiliar environment. It is a prerequisite for many robot application scenarios, such as path planning. , collision-free navigation, environmental perception, etc., and visual SLAM refers to the positioning and mapping method with the camera as the main data acquisition sensor. The picture has rich texture information, which can generate the semantic information of the environment by combining with deep learning. [0003] The feature-based method can only create a sparse point cloud map without environmental semantic information. Although the direct method can create a ...

AI Technical Summary

Problems solved by technology

[0003] The feature-based method can only create a sparse point cloud map without environmental semantic information. Although the direct method can create a semi-dense or dense map, it is accurate to directly use the optical flow to calculate the entire image, especially when the image size is large. disadvantages such as low precision and large amount of calculation

Images