Coupled Convolutional Neural Network-Based Detail Injection Method for Hyperspectral and Multispectral Image Fusion

• Main Authors: Xiaochen Lu, Dezheng Yang, Fengde Jia, Yifeng Zhao
• Format: Article
• Language: English
• Published: MDPI AG 2021-12-01
• Series: Applied Sciences
• Subjects: convolutional neural network; hyper-sharpening; hyperspectral; image fusion; multispectral
• Online Access: https://www.mdpi.com/2076-3417/11/1/288

In this paper, a detail-injection method based on a coupled convolutional neural network (CNN) is proposed for hyperspectral (HS) and multispectral (MS) image fusion with the goal of enhancing the spatial resolution of HS images. Owing to the excellent performance in spectral fidelity of the detail-injection model and the image spatial–spectral feature exploration ability of CNN, the proposed method utilizes a couple of CNN networks as the feature extraction method and learns details from the HS and MS images individually. By appending an additional convolutional layer, both the extracted features of two images are concatenated to predict the missing details of the anticipated HS image. Experiments on simulated and real HS and MS data show that compared with some state-of-the-art HS and MS image fusion methods, our proposed method achieves better fusion results, provides excellent spectrum preservation ability, and is easy to implement.