Multi-modal three-dimensional point cloud segmentation system and method

Abstract

The invention discloses a multi-modal three-dimensional point cloud segmentation system and method. According to the invention, the good fusion of modal data can be realized; a priori mask is introduced, robustness of an obtained scene segmentation result is improved, and the high segmentation precision is obtained. For different scenes, such as toilets, meeting rooms and offices, a good prediction result can be obtained, and the model has good generalization. For an unused skeleton network used for extracting point cloud features, a feature and decision fusion module can be attempted to be utilized, and the precision is improved; if calculation conditions allow, more points can be tried, and a larger area can be utilized, for example, the number of used points and the size of a scene areaare increased by the same multiple, so that the receptive field of the whole model is improved, and the perceptual capacity of the model to the whole scene is improved.

Description

technical field [0001] The invention relates to the technical fields of computer vision and computer graphics, in particular to a multi-modal three-dimensional point cloud segmentation system and method. Background technique [0002] With the rapid development of 3D acquisition technology, related sensors are becoming more and more common in our lives, such as various 3D scanning devices, lidar and RGB-D cameras, etc. 3D point cloud data is used in many machine vision tasks, such as autonomous driving, robot navigation, virtual reality and augmented reality, etc. In addition, point cloud related technologies play an important role in medical image processing, computer graphics and other fields. For visual tasks, images are easily affected by ambient lighting and shooting angles, and the spatial structure information of objects is lost to a certain extent; while point cloud data can contain geometric information of specific scenes in 3D space, and are not easily affected by ...

AI Technical Summary

Problems solved by technology

However, there are still the following problems: 1) point cloud data is naturally sparse and susceptible to noise interference, and the learning of point clouds needs to ensure a certain degree of robustness; 2) the current point cloud processing related technologies can be applied to limited scenarios, due to Its calculation consumes a lot, and it cannot directly process large-scale real point cloud scenes; 3) At the same time as point cloud data acquisition, image data is relatively easy to obtain. The current methods are shallow fusion at the input level, which lacks efficiency. method of information fusion

Images