Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure
Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2021-01-01
|
Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9461404/ |
id |
doaj-d6ab26630449416e96c99be50d26bbeb |
---|---|
record_format |
Article |
spelling |
doaj-d6ab26630449416e96c99be50d26bbeb2021-07-26T23:00:09ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352021-01-01147066708410.1109/JSTARS.2021.30909589461404Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese StructureXin Jin0https://orcid.org/0000-0003-2211-2006Shanshan Huang1Qian Jiang2https://orcid.org/0000-0003-3097-0721Shin-Jye Lee3https://orcid.org/0000-0003-4265-5016Liwen Wu4Shaowen Yao5https://orcid.org/0000-0003-1516-4246School of Software, Yunnan University, Kunming, ChinaSchool of Software, Yunnan University, Kunming, ChinaSchool of Software, Yunnan University, Kunming, ChinaInstitute of Technology Management, National Chiao Tung University, Hsinchu, ChinaSchool of Software, Yunnan University, Kunming, ChinaSchool of Software, Yunnan University, Kunming, ChinaRemote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semisupervised RSIF method based on the multiscale conditional generative adversarial networks by combining the multiskip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multispectral images to fuse them without a ground truth; the adopted multiskip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-to-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth.https://ieeexplore.ieee.org/document/9461404/Conditional generative adversarial network (cGAN)deep learning (DL)image fusionloss functionremote sensing image fusion (RSIF) |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Xin Jin Shanshan Huang Qian Jiang Shin-Jye Lee Liwen Wu Shaowen Yao |
spellingShingle |
Xin Jin Shanshan Huang Qian Jiang Shin-Jye Lee Liwen Wu Shaowen Yao Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Conditional generative adversarial network (cGAN) deep learning (DL) image fusion loss function remote sensing image fusion (RSIF) |
author_facet |
Xin Jin Shanshan Huang Qian Jiang Shin-Jye Lee Liwen Wu Shaowen Yao |
author_sort |
Xin Jin |
title |
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure |
title_short |
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure |
title_full |
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure |
title_fullStr |
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure |
title_full_unstemmed |
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure |
title_sort |
semisupervised remote sensing image fusion using multiscale conditional generative adversarial network with siamese structure |
publisher |
IEEE |
series |
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
issn |
2151-1535 |
publishDate |
2021-01-01 |
description |
Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semisupervised RSIF method based on the multiscale conditional generative adversarial networks by combining the multiskip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multispectral images to fuse them without a ground truth; the adopted multiskip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-to-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth. |
topic |
Conditional generative adversarial network (cGAN) deep learning (DL) image fusion loss function remote sensing image fusion (RSIF) |
url |
https://ieeexplore.ieee.org/document/9461404/ |
work_keys_str_mv |
AT xinjin semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure AT shanshanhuang semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure AT qianjiang semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure AT shinjyelee semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure AT liwenwu semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure AT shaowenyao semisupervisedremotesensingimagefusionusingmultiscaleconditionalgenerativeadversarialnetworkwithsiamesestructure |
_version_ |
1721280577777172480 |