RGB-D image semantic segmentation method based on multi-modal adaptive convolution

Abstract

The invention relates to an RGB-D image semantic segmentation method based on multi-modal adaptive convolution. The method comprises the steps that an encoding module extracts RGB image features and depth image features; the RGB features and the depth features are sent to a fusion module for fusion; the method comprises the following steps: firstly, inputting multi-modal features into a multi-modal adaptive convolution generation module, and calculating two multi-modal adaptive convolution kernels with different scales; then, enabling the multi-modal feature fusion module to carry out depth separable convolution operation on the RGB features and the depth features and an adaptive convolution kernel to obtain adaptive convolution fusion features; splicing the fusion features with the RGB features and the depth features to obtain final fusion features; enabling the decoding module to perform continuous up-sampling on the final fusion feature, and obtaining a semantic segmentation resultthrough convolution operation. According to the invention, multi-modal features are interacted cooperatively through adaptive convolution, and convolution kernel parameters are dynamically adjusted according to an input multi-modal image, so that the method is more flexible than a traditional convolution kernel with fixed parameters.

Description

technical field [0001] The invention relates to the fields of image semantic segmentation and deep learning, in particular to a convolutional neural network-based RGB-D image semantic segmentation method. Background technique [0002] Image semantic segmentation is one of the basic tasks in the field of artificial intelligence and computer vision. Its purpose is to identify the semantic category of each pixel in the image according to the image content. As the basis of image and video understanding, semantic segmentation is widely used in intelligent fields such as autonomous driving and robot navigation. [0003] With the wide application of deep learning in computer vision, deep convolutional neural network has become the most effective method in the field of computer vision. In 2015, the fully convolutional neural network pioneered the use of deep learning for end-to-end image feature extraction and pixel semantic classification, which has greatly improved performance an...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a new RGB-D image semantic segmentation method and system based on multimodal adaptive convolution in order to solve the problem of low accuracy of the existing RGB-D image semantic segmentation method

Images