Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation
With the ability for all-day, all-weather acquisition, synthetic aperture radar (SAR) remote sensing is an important technique in modern Earth observation. However, the interpretation of SAR images is a highly challenging task, even for well-trained experts, due to the imaging principle of SAR image...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2021-09-01
|
Series: | Remote Sensing |
Subjects: | |
Online Access: | https://www.mdpi.com/2072-4292/13/18/3575 |
id |
doaj-855c24149f9b43a8a5772b851cb39212 |
---|---|
record_format |
Article |
spelling |
doaj-855c24149f9b43a8a5772b851cb392122021-09-26T01:15:44ZengMDPI AGRemote Sensing2072-42922021-09-01133575357510.3390/rs13183575Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image TranslationJie Guo0Chengyu He1Mingjin Zhang2Yunsong Li3Xinbo Gao4Bangyu Song5Key Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, ChinaKey Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, ChinaKey Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, ChinaKey Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, ChinaKey Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, ChinaSchool of Integrated Circuit Science and Engineering, Beihang University, Beijing 100191, ChinaWith the ability for all-day, all-weather acquisition, synthetic aperture radar (SAR) remote sensing is an important technique in modern Earth observation. However, the interpretation of SAR images is a highly challenging task, even for well-trained experts, due to the imaging principle of SAR images and the high-frequency speckle noise. Some image-to-image translation methods are used to convert SAR images into optical images that are closer to what we perceive through our eyes. There exist two weaknesses in these methods: (1) these methods are not designed for an SAR-to-optical translation task, thereby losing sight of the complexity of SAR images and the speckle noise. (2) The same convolution filters in a standard convolution layer are utilized for the whole feature maps, which ignore the details of SAR images in each window and generate images with unsatisfactory quality. In this paper, we propose an edge-preserving convolutional generative adversarial network (EPCGAN) to enhance the structure and aesthetics of the output image by leveraging the edge information of the SAR image and implementing content-adaptive convolution. The proposed edge-preserving convolution (EPC) decomposes the content of the convolution input into texture components and content components and then generates a content-adaptive kernel to modify standard convolutional filter weights for the content components. Based on the EPC, the EPCGAN is presented for SAR-to-optical image translation. It uses a gradient branch to assist in the recovery of structural image information. Experiments on the SEN1-2 dataset demonstrated that the proposed method can outperform other SAR-to-optical methods by recovering more structures and yielding a superior evaluation index.https://www.mdpi.com/2072-4292/13/18/3575SAR-to-optical image translationdeep learninggenerative adversarial networksedge-preserving convolution |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Jie Guo Chengyu He Mingjin Zhang Yunsong Li Xinbo Gao Bangyu Song |
spellingShingle |
Jie Guo Chengyu He Mingjin Zhang Yunsong Li Xinbo Gao Bangyu Song Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation Remote Sensing SAR-to-optical image translation deep learning generative adversarial networks edge-preserving convolution |
author_facet |
Jie Guo Chengyu He Mingjin Zhang Yunsong Li Xinbo Gao Bangyu Song |
author_sort |
Jie Guo |
title |
Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation |
title_short |
Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation |
title_full |
Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation |
title_fullStr |
Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation |
title_full_unstemmed |
Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation |
title_sort |
edge-preserving convolutional generative adversarial networks for sar-to-optical image translation |
publisher |
MDPI AG |
series |
Remote Sensing |
issn |
2072-4292 |
publishDate |
2021-09-01 |
description |
With the ability for all-day, all-weather acquisition, synthetic aperture radar (SAR) remote sensing is an important technique in modern Earth observation. However, the interpretation of SAR images is a highly challenging task, even for well-trained experts, due to the imaging principle of SAR images and the high-frequency speckle noise. Some image-to-image translation methods are used to convert SAR images into optical images that are closer to what we perceive through our eyes. There exist two weaknesses in these methods: (1) these methods are not designed for an SAR-to-optical translation task, thereby losing sight of the complexity of SAR images and the speckle noise. (2) The same convolution filters in a standard convolution layer are utilized for the whole feature maps, which ignore the details of SAR images in each window and generate images with unsatisfactory quality. In this paper, we propose an edge-preserving convolutional generative adversarial network (EPCGAN) to enhance the structure and aesthetics of the output image by leveraging the edge information of the SAR image and implementing content-adaptive convolution. The proposed edge-preserving convolution (EPC) decomposes the content of the convolution input into texture components and content components and then generates a content-adaptive kernel to modify standard convolutional filter weights for the content components. Based on the EPC, the EPCGAN is presented for SAR-to-optical image translation. It uses a gradient branch to assist in the recovery of structural image information. Experiments on the SEN1-2 dataset demonstrated that the proposed method can outperform other SAR-to-optical methods by recovering more structures and yielding a superior evaluation index. |
topic |
SAR-to-optical image translation deep learning generative adversarial networks edge-preserving convolution |
url |
https://www.mdpi.com/2072-4292/13/18/3575 |
work_keys_str_mv |
AT jieguo edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation AT chengyuhe edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation AT mingjinzhang edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation AT yunsongli edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation AT xinbogao edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation AT bangyusong edgepreservingconvolutionalgenerativeadversarialnetworksforsartoopticalimagetranslation |
_version_ |
1716869159063650304 |