Three-dimensional point cloud scene segmentation method and system fusing image features

Abstract

The invention provides a three-dimensional point cloud scene segmentation method and system fusing image features, relates to the technical field of computer vision, and can realize effective fusion of a two-dimensional image and a three-dimensional point cloud and accurate segmentation of a three-dimensional scene. The method comprises the following steps: S1, acquiring two-dimensional data, point cloud data and depth data including an image, and calculating an association relationship between a scene image and a point cloud according to the acquired data; s2, performing feature extraction on the two-dimensional data to obtain a high-dimensional to-be-fused feature map; s3, fusing the to-be-fused feature map and the point cloud data according to a fusion strategy to obtain fused point cloud data; the fusion strategy comprises the following steps: by searching for a pixel adjacent to a certain point cloud data, warping a feature corresponding to the pixel to the point cloud data; and S4, inputting the fused point cloud data into the three-dimensional segmentation network for feature extraction, thereby obtaining required global and local semantic information. The technical scheme provided by the invention is suitable for a three-dimensional point cloud processing process.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method and system for segmenting a three-dimensional point cloud scene by fusing image features. Background technique [0002] With the development of artificial intelligence, people are eager for machines to obtain real-world information like humans, and intelligently analyze these data and give corresponding judgments. As an important technology for machine perception of the real world, computer vision has always been a research hotspot in the field of artificial intelligence. Semantic segmentation of 3D scenes, as a basic task of computer vision, has important research value and application prospects, and its required precision is increasing day by day. [0003] Scene data includes two-dimensional data represented by two-dimensional images and three-dimensional data represented by RGB-D depth maps and three-dimensional data. Two-dimensional images are affected by l...

AI Technical Summary

Problems solved by technology

Two-dimensional images are affected by lighting conditions, complex environments, etc., and the imaging quality will decrease to a certain extent. At the same time, object occlusion is also an important factor that two-dimensional images cannot better describe the real world.

First of all, point cloud is data in three-dimensional space, and its spatial distribution characteristics and high-dimensional information of points are difficult to capture. Second, three-dimensional point cloud is composed of disordered points, while two-dimensional images are ordered pixels. Two different types It is difficult to integrate data or features

Compared with the regular grid structure of image data, the sparseness and unstructured nature of point clouds make it impossible to extract features directly through CNN

Images