When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework
In recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of r...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2020-10-01
|
Series: | Remote Sensing |
Subjects: | |
Online Access: | https://www.mdpi.com/2072-4292/12/20/3276 |
id |
doaj-4f0a8b60d07541df9fd9e3755d844044 |
---|---|
record_format |
Article |
spelling |
doaj-4f0a8b60d07541df9fd9e3755d8440442020-11-25T03:57:22ZengMDPI AGRemote Sensing2072-42922020-10-01123276327610.3390/rs12203276When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning FrameworkZhicheng Zhao0Ze Luo1Jian Li2Can Chen3Yingchao Piao4Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, ChinaComputer Network Information Center, Chinese Academy of Sciences, Beijing 100190, ChinaComputer Network Information Center, Chinese Academy of Sciences, Beijing 100190, ChinaComputer Network Information Center, Chinese Academy of Sciences, Beijing 100190, ChinaComputer Network Information Center, Chinese Academy of Sciences, Beijing 100190, ChinaIn recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of remote sensing image datasets is generally small. In addition, many problems related to small objects and complex backgrounds arise in remote sensing image scenes, presenting great challenges for CNN-based recognition methods. In this article, to improve the feature extraction ability and generalization ability of such models and to enable better use of the information contained in the original remote sensing images, we introduce a multitask learning framework which combines the tasks of self-supervised learning and scene classification. Unlike previous multitask methods, we adopt a new mixup loss strategy to combine the two tasks with dynamic weight. The proposed multitask learning framework empowers a deep neural network to learn more discriminative features without increasing the amounts of parameters. Comprehensive experiments were conducted on four representative remote sensing scene classification datasets. We achieved state-of-the-art performance, with average accuracies of 94.21%, 96.89%, 99.11%, and 98.98% on the NWPU, AID, UC Merced, and WHU-RS19 datasets, respectively. The experimental results and visualizations show that our proposed method can learn more discriminative features and simultaneously encode orientation information while effectively improving the accuracy of remote sensing scene classification.https://www.mdpi.com/2072-4292/12/20/3276self-supervisedmultitaskCNNscene classificationNWPUdeep learning |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Zhicheng Zhao Ze Luo Jian Li Can Chen Yingchao Piao |
spellingShingle |
Zhicheng Zhao Ze Luo Jian Li Can Chen Yingchao Piao When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework Remote Sensing self-supervised multitask CNN scene classification NWPU deep learning |
author_facet |
Zhicheng Zhao Ze Luo Jian Li Can Chen Yingchao Piao |
author_sort |
Zhicheng Zhao |
title |
When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework |
title_short |
When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework |
title_full |
When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework |
title_fullStr |
When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework |
title_full_unstemmed |
When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on A Multitask Learning Framework |
title_sort |
when self-supervised learning meets scene classification: remote sensing scene classification based on a multitask learning framework |
publisher |
MDPI AG |
series |
Remote Sensing |
issn |
2072-4292 |
publishDate |
2020-10-01 |
description |
In recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of remote sensing image datasets is generally small. In addition, many problems related to small objects and complex backgrounds arise in remote sensing image scenes, presenting great challenges for CNN-based recognition methods. In this article, to improve the feature extraction ability and generalization ability of such models and to enable better use of the information contained in the original remote sensing images, we introduce a multitask learning framework which combines the tasks of self-supervised learning and scene classification. Unlike previous multitask methods, we adopt a new mixup loss strategy to combine the two tasks with dynamic weight. The proposed multitask learning framework empowers a deep neural network to learn more discriminative features without increasing the amounts of parameters. Comprehensive experiments were conducted on four representative remote sensing scene classification datasets. We achieved state-of-the-art performance, with average accuracies of 94.21%, 96.89%, 99.11%, and 98.98% on the NWPU, AID, UC Merced, and WHU-RS19 datasets, respectively. The experimental results and visualizations show that our proposed method can learn more discriminative features and simultaneously encode orientation information while effectively improving the accuracy of remote sensing scene classification. |
topic |
self-supervised multitask CNN scene classification NWPU deep learning |
url |
https://www.mdpi.com/2072-4292/12/20/3276 |
work_keys_str_mv |
AT zhichengzhao whenselfsupervisedlearningmeetssceneclassificationremotesensingsceneclassificationbasedonamultitasklearningframework AT zeluo whenselfsupervisedlearningmeetssceneclassificationremotesensingsceneclassificationbasedonamultitasklearningframework AT jianli whenselfsupervisedlearningmeetssceneclassificationremotesensingsceneclassificationbasedonamultitasklearningframework AT canchen whenselfsupervisedlearningmeetssceneclassificationremotesensingsceneclassificationbasedonamultitasklearningframework AT yingchaopiao whenselfsupervisedlearningmeetssceneclassificationremotesensingsceneclassificationbasedonamultitasklearningframework |
_version_ |
1724461353529245696 |