Semantically-driven camera positioning and map reconstruction method and system

Abstract

The invention discloses a semantically-driven camera positioning and map reconstruction method, and belongs to the technical field of computer vision. The method comprises the following steps: firstly, performing semantic segmentation on feature points of a current frame image; matching all the feature points in the current frame and the key frame by adopting a similar matching method according tothe similarity and the semantic category to obtain a matching pair; initializing the posture of the camera through all matching in the current frame and the key frame; updating the feature point matching pairs by adopting a three-dimensional projection method in combination with semantic judgment; updating all the feature point matching pairs by utilizing attitude minimization; and finally, constructing a three-dimensional map by utilizing the camera attitude. The invention further discloses a semantic-driven camera positioning and map reconstruction system. According to the technical schemeprovided by the invention, not only is a plurality of processing performed in a camera positioning stage, but also point cloud constraint is performed in a reconstruction stage, so that semantic segmentation is more tightly combined with a camera positioning and reconstruction system, and a more accurate positioning result and a more perfect reconstruction effect are obtained.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and more specifically relates to a semantic-driven camera positioning and map reconstruction method. Background technique [0002] At present, the camera localization and reconstruction technology is not combined with the semantic segmentation technology or not tightly combined. [0003] For related algorithms that do not combine semantic segmentation, on the one hand, it is more difficult to cope with various environments, such as dynamic scenes and weak texture scenes. On the other hand, the map models reconstructed by these algorithms are often composed of point clouds or landmarks, which are all maps based on geometric information, so they cannot provide any high-level understanding of the surrounding environment. [0004] For the related algorithms combined with semantic segmentation, the recognition objects are generally labeled with categories, and some optimizations are made to r...

AI Technical Summary

Problems solved by technology

[0003] For related algorithms that do not combine semantic segmentation, on the one hand, it is difficult to cope with various environments, such as dynamic scenes and weak texture scenes

On the other hand, the map models reconstructed by these algorithms are often composed of point clouds or landmarks, which are all maps based on geometric information, so they cannot provide any high-level understanding of the surrounding environment.

[0004] For the related algorithms combined with semantic segmentation, the recognition objects are generally labeled with categories, and some optimizations are made to remove the influence of dynamic objects, but the results of semantic segmentation are not fully utilized, and semantic segmentation is closely integrated into positioning and map reconstruction technology system

Images