Land cover classification method based on deep fusion of multi-modal remote sensing data

Abstract

The invention relates to a land cover classification method based on deep fusion of multi-modal remote sensing data. The method comprises the following steps: (1) constructing a remote sensing image semantic segmentation network based on multi-modal information fusion, wherein the network comprises an encoder for extracting ground feature features, a depth feature fusion module, a spatial pyramid module and an up-sampling decoder, and the depth feature fusion module comprises an ACF3 module and a CACF3 module which are used for simultaneously fusing three modal information of RGB, DSM and NDVI, the ACF3 module is a self-attention convolution fusion module based on transformer and convolution, and the CACF3 module is a cross-modal convolution fusion module based on transformer and convolution; (2) training the network constructed in the step (1); (3) utilizing the network model trained in the step (2) to predict the remote sensing image ground object category. Compared with a conventional method, the earth surface coverage classification method based on multi-modal remote sensing data deep fusion has the advantages that the precision improvement effect on earth surface classification tasks is remarkable, and the application prospect is wide.

Description

technical field [0001] The invention belongs to the technical field of remote sensing, and relates to a land cover classification method based on deep fusion of multimodal remote sensing data. Background technique [0002] The classification of surface objects (ground objects) is an important basis for the application of remote sensing image analysis. Nowadays, the continuous observation of the surface by multiple sensors has given birth to multi-scale, multi-temporal, multi-azimuth, and multi-level surface remote sensing images, which provide richer data information for the accurate description of ground objects. Since it is essentially the observation of the same ground object, although there is a certain gap between different modal information, there is still a complementary characteristic of information between multi-source remote sensing images. Therefore, using multiple remote sensing information sources to classify ground objects can achieve higher accuracy than sing...

AI Technical Summary

Problems solved by technology

However, none of the existing methods has done more exploration in the field of deep feature fusion. For example, the densely connected transformation network (DCST) uses the transformer as a feature extractor to try to replace the encoder structure in the classic convolutional network. ; For example, U-shaped network-based transformation network (TransUNet) regards transformer as a high-dimensional feature extraction module, and further processes the features extracted by classic encoders; such as U-shaped network-based pure attention transformation network (Swin-Unet) Following the U-shaped network, the entire network structure is used for feature extraction and type prediction by transformer

These methods regard the transformer as a feature extraction structure and do not utilize it in the field of feature fusion

Images