Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification
Deep learning models combining spectral and spatial features have been proven to be effective for hyperspectral image (HSI) classification. However, most spatial feature integration methods only consider a single input spatial scale regardless of various shapes and sizes of objects over the image pl...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2019-04-01
|
Series: | Sensors |
Subjects: | |
Online Access: | https://www.mdpi.com/1424-8220/19/7/1714 |
id |
doaj-4fdc9d8cc0784168ab88e7b30a5cb05f |
---|---|
record_format |
Article |
spelling |
doaj-4fdc9d8cc0784168ab88e7b30a5cb05f2020-11-24T23:41:41ZengMDPI AGSensors1424-82202019-04-01197171410.3390/s19071714s19071714Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image ClassificationSimin Li0Xueyu Zhu1Jie Bao2Department of Electronic Engineering, Tsinghua University, Beijing 100084, ChinaDepartment of Mathematics, University of Iowa, Iowa City, IA 52242, USADepartment of Electronic Engineering, Tsinghua University, Beijing 100084, ChinaDeep learning models combining spectral and spatial features have been proven to be effective for hyperspectral image (HSI) classification. However, most spatial feature integration methods only consider a single input spatial scale regardless of various shapes and sizes of objects over the image plane, leading to missing scale-dependent information. In this paper, we propose a hierarchical multi-scale convolutional neural networks (CNNs) with auxiliary classifiers (HMCNN-AC) to learn hierarchical multi-scale spectral–spatial features for HSI classification. First, to better exploit the spatial information, multi-scale image patches for each pixel are generated at different spatial scales. These multi-scale patches are all centered at the same central spectrum but with shrunken spatial scales. Then, we apply multi-scale CNNs to extract spectral–spatial features from each scale patch. The obtained multi-scale convolutional features are considered as structured sequential data with spectral–spatial dependency, and a bidirectional LSTM is proposed to capture the correlation and extract a hierarchical representation for each pixel. To better train the whole network, weighted auxiliary classifiers are employed for the multi-scale CNNs and optimized together with the main loss function. Experimental results on three public HSI datasets demonstrate the superiority of our proposed framework over some state-of-the-art methods.https://www.mdpi.com/1424-8220/19/7/1714hyperspectral image (HSI) classificationconvolutional neural networks (CNNs)bidirectional LSTMmulti-scale features |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Simin Li Xueyu Zhu Jie Bao |
spellingShingle |
Simin Li Xueyu Zhu Jie Bao Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification Sensors hyperspectral image (HSI) classification convolutional neural networks (CNNs) bidirectional LSTM multi-scale features |
author_facet |
Simin Li Xueyu Zhu Jie Bao |
author_sort |
Simin Li |
title |
Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification |
title_short |
Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification |
title_full |
Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification |
title_fullStr |
Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification |
title_full_unstemmed |
Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification |
title_sort |
hierarchical multi-scale convolutional neural networks for hyperspectral image classification |
publisher |
MDPI AG |
series |
Sensors |
issn |
1424-8220 |
publishDate |
2019-04-01 |
description |
Deep learning models combining spectral and spatial features have been proven to be effective for hyperspectral image (HSI) classification. However, most spatial feature integration methods only consider a single input spatial scale regardless of various shapes and sizes of objects over the image plane, leading to missing scale-dependent information. In this paper, we propose a hierarchical multi-scale convolutional neural networks (CNNs) with auxiliary classifiers (HMCNN-AC) to learn hierarchical multi-scale spectral–spatial features for HSI classification. First, to better exploit the spatial information, multi-scale image patches for each pixel are generated at different spatial scales. These multi-scale patches are all centered at the same central spectrum but with shrunken spatial scales. Then, we apply multi-scale CNNs to extract spectral–spatial features from each scale patch. The obtained multi-scale convolutional features are considered as structured sequential data with spectral–spatial dependency, and a bidirectional LSTM is proposed to capture the correlation and extract a hierarchical representation for each pixel. To better train the whole network, weighted auxiliary classifiers are employed for the multi-scale CNNs and optimized together with the main loss function. Experimental results on three public HSI datasets demonstrate the superiority of our proposed framework over some state-of-the-art methods. |
topic |
hyperspectral image (HSI) classification convolutional neural networks (CNNs) bidirectional LSTM multi-scale features |
url |
https://www.mdpi.com/1424-8220/19/7/1714 |
work_keys_str_mv |
AT siminli hierarchicalmultiscaleconvolutionalneuralnetworksforhyperspectralimageclassification AT xueyuzhu hierarchicalmultiscaleconvolutionalneuralnetworksforhyperspectralimageclassification AT jiebao hierarchicalmultiscaleconvolutionalneuralnetworksforhyperspectralimageclassification |
_version_ |
1725505948278063104 |