Semantic SLAM system and method based on joint constraints

Abstract

The invention provides a semantic SLAM system and method based on joint constraints. The objective of the invention is to solve the problems of inaccurate camera pose calculation under the condition of unstable pixel depth values and incapability of calculating the camera pose when a dynamic target occupies most of the space of a camera field of view, improve the accuracy of camera pose estimationthrough a depth constraint method, and improve the integrity of a camera track through an epipolar constraint method. The implementation method comprises the steps that a data acquisition module acquires an image sequence; a neural network module acquires a detection image and an instance segmentation image; a joint constraint module acquires different feature point category sets; a data fusion module acquires a static target instance segmentation image and a dynamic target instance segmentation image;a visual front-end module acquires the pose of the depth camera and a road sign point set ina three-dimensional space; a rear-end optimization module acquires a globally optimal depth camera pose and a road sign point; and a semantic map module acquires a semantic point cloud map.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and further relates to a semantic SLAM system and method based on joint constraints, which can be used for pose estimation of cameras and construction of semantic maps in complex and highly dynamic environments. Background technique [0002] At the same time, the positioning and map construction system SLAM plays an important role in the autonomous navigation and obstacle avoidance of the unmanned system. In the past three decades, the SLAM system has developed rapidly, and its main goal is the autonomous exploration of the unknown environment by the unmanned system. In the process of positioning itself accurately, it can build a map of the environment. However, the maps built by traditional SLAM systems only contain low-level geometric features such as points, lines, and surfaces in the environment. For future unmanned systems, maps that only contain simple spatial information are diffic...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art, and propose a semantic SLAM system and method based on joint constraints, which are used to solve the inaccurate calculation of the camera pose when the pixel depth value is unstable and when the dynamic target occupies the camera. The problem that the camera pose cannot be calculated when viewing most of the space, in order to improve the accuracy of the camera pose and the integrity of the camera trajectory, and at the same time solve the problem that objects with motion properties cannot be constructed in the point cloud map when they are stationary, and obtain more Accurate point cloud map

Images