PercepPan: Towards Unsupervised Pan-Sharpening Based on Perceptual Loss

• Main Authors: Changsheng Zhou, Jiangshe Zhang, Junmin Liu, Chunxia Zhang, Rongrong Fei, Shuang Xu
• Format: Article
• Language: English
• Published: MDPI AG 2020-07-01
• Series: Remote Sensing
• Subjects: pan-sharpening; perceptual loss; auto-encoder; generative adversarial networks; unsupervised learning
• Online Access: https://www.mdpi.com/2072-4292/12/14/2318

In the literature of pan-sharpening based on neural networks, high resolution multispectral images as ground-truth labels generally are unavailable. To tackle the issue, a common method is to degrade original images into a lower resolution space for supervised training under the Wald’s protocol. In this paper, we propose an unsupervised pan-sharpening framework, referred to as “perceptual pan-sharpening”. This novel method is based on auto-encoder and perceptual loss, and it does not need the degradation step for training. For performance boosting, we also suggest a novel training paradigm, called “first supervised pre-training and then unsupervised fine-tuning”, to train the unsupervised framework. Experiments on the QuickBird dataset show that the framework with different generator architectures could get comparable results with the traditional supervised counterpart, and the novel training paradigm performs better than random initialization. When generalizing to the IKONOS dataset, the unsupervised framework could still get competitive results over the supervised ones.