Multispectral image and panchromatic image fusion method based on generative adversarial network

Abstract

The invention discloses a multispectral image and panchromatic image fusion method based on a generative adversarial network. The method comprises: firstly, collecting multispectral image and panchromatic image data sets, carrying out registration processing on images, and dividing the data sets into a training set and a test set; constructing a feature extraction network, and inputting the feature extraction network as a panchromatic image; secondly, constructing a generative adversarial network, optimizing a discriminator by utilizing a Wassertein distance, inputting a multispectral image, outputting a fused high-resolution multispectral image, finally training the generative adversarial network, and performing testing by utilizing a test set. Particularly, the method does not need an additional processing flow, and is an end-to-end method for realizing multispectral image fusion. Moreover, according to the method, a large amount of data is used as a fusion mapping relation for driving learning, and for most images with different data sets, the method can better maintain spectral information of an original multispectral image while spatial detail information of the fused image isinjected.

Description

technical field [0001] The invention relates to a remote sensing image processing method, in particular to an image fusion method based on a generating confrontation network. Background technique [0002] Remote sensing satellites can obtain multispectral images with low spatial resolution and panchromatic images with high spatial resolution by using spectral sensors. However, the panchromatic image has higher spatial resolution, which can fully reflect the position and edge features of ground objects, which is conducive to the accurate positioning of ground objects, but its The spectral resolution is low; therefore, research issues on the fusion of multispectral images and panchromatic images have attracted great attention; the fused images have high spectral resolution and spatial resolution at the same time, which is more convenient and comprehensive It provides the possibility to understand the environment and natural resources; it is often used in surveying and mapping...

AI Technical Summary

Problems solved by technology

[0002] Remote sensing satellites can obtain multispectral images with low spatial resolution and panchromatic images with high spatial resolution by using spectral sensors. However, the panchromatic image has higher spatial resolution, which can fully reflect the position and edge features of ground objects, which is conducive to the accurate positioning of ground objects, but its The spectral resolution is low; therefore, research issues on the fusion of multispectral images and panchromatic images have attracted great attention; the fused images have high spectral resolution and spatial resolution at the same time, which is more convenient and comprehensive It provides the possibility to understand the environment and natural resources; it is often used in surveying and mapping, object detection, vegetation classification or object classification, weather forecasting and other remote sensing data applications

[0003] Traditional methods are mainly divided into four categories: component substitution-based methods, multi-resolution analysis-based methods, model-based methods, and super-resolution-based methods; in component substitution-based algorithms, multispectral images are usually transformed into color spaces , to separate the spatial and spectral information of the image, and then use the panchromatic image to achieve component replacement, and inverse transform the color space to obtain the final fusion image; such methods usually provide high-quality spatial details and high computational efficiency, but they usually Spectral distortion will also be introduced in pan-sharpening; common image fusion methods based on component substitution include IHS transformation fusion method, Brovey, GS, PCA transformation fusion method, etc.; algorithms based on multi-resolution analysis realize multiple The spatial detail information can be obtained by distinguishing and decomposing into multispectral images to obtain the final fused image; this method can not only accurately extract features from decomposed images of different scales, but also reduce halo and aliasing in the fusion process. overlap artifacts, but require high computational cost; common image fusion methods based on multi-resolution analysis include wavelet transform (ATWT), Laplacian pyramid decomposition, smoothing filter intensity modulation (SFIM) and other methods; model-based The algorithm first creates a relationship model between the fused image and the panchromatic / multispectral image, and then optimizes the model to obtain the final fused image; the model-based image fusion method solves the problem of spectral distortion very well. However, the solution process of the model is complicated and the time efficiency is low; common model methods include P+XS algorithm, sparse coding algorithm, prior-based algorithm, etc. Among them, the sparse coding method learns the corresponding sparseness by constructing a high- and low-resolution dictionary set. coefficients, so as to obtain the final fusion image; although this algorithm can achieve a satisfactory fusion effect, it needs a large number of training images to generate a large-scale dictionary; from the analysis of the above methods, the traditional method has the characteristics of simple framework, and realizes In order to enhance the spatial resolution of multispectral images to a certain extent, however, it is often accompanied by severe spectral distortion

Images