Deep Neural Network Structured Sparse Coding for Online Processing
Sparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, has become a mature class of methods with good efficiency in various areas, but it faces limitations in immediate processing such as real-time video denoising. Unsupervised deep neura...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2018-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/8542719/ |
id |
doaj-98134795c1a44da086cf55b4edebebba |
---|---|
record_format |
Article |
spelling |
doaj-98134795c1a44da086cf55b4edebebba2021-03-29T21:35:21ZengIEEEIEEE Access2169-35362018-01-016747787479110.1109/ACCESS.2018.28825318542719Deep Neural Network Structured Sparse Coding for Online ProcessingHaoli Zhao0https://orcid.org/0000-0002-4004-509XShuxue Ding1Xiang Li2Huakun Huang3https://orcid.org/0000-0003-2853-8892The School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanThe School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanThe School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanThe School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanSparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, has become a mature class of methods with good efficiency in various areas, but it faces limitations in immediate processing such as real-time video denoising. Unsupervised deep neural network structured sparse coding (DNN-SC) algorithms can enhance the efficiency of iterative sparse coding algorithms to achieve the goal. In this paper, we first propose a sparse coding algorithm by adding the idea “weighted" in the iterative shrinkage thresholding algorithm (ISTA), named WISTA, which can enjoy the benefit of the l<sub>p</sub> norm (0 <; p <; 1) sparsity constraint. Then, we propose two novel DNN-SC algorithms by combining deep learning with WISTA and the iterative half thresholding algorithm (IHTA), which is the l<sub>0.5</sub> norm sparse coding algorithm. Furthermore, we present that by changing the loss function, the DNN can be learned supervisedly and unsupervisedly. Unsupervised learning is the key to ensure the DNN to be learned online during processing, which enables the use of the DNN-SC algorithms in applications lacking labels for signals. Synthetic data experiments show that WISTA can outperform ISTA and IHTA. Moreover, the DNNstructured WISTA can successfully achieve converged results of WISTA. In real-world data experiments, the procedure of utilizing DNN-SC algorithms in image denoising is first presented. All DNN-SC algorithms can accelerate at least 45 times while maintaining PSNR results compared with their corresponding sparse coding algorithms. Finally, the strategy of utilizing DNN-SC algorithms in real-time video denoising is presented. The video-denoising experiments show that the DNN-structured ISTA and WISTA can conduct real-time video denoising for 25 frames/s 360 × 480 pixels gray-scaled videos.https://ieeexplore.ieee.org/document/8542719/Sparse codingdeep neural networkweighted iterative shrinkage thresholding algorithmunsupervised learningreal-time video denoising |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Haoli Zhao Shuxue Ding Xiang Li Huakun Huang |
spellingShingle |
Haoli Zhao Shuxue Ding Xiang Li Huakun Huang Deep Neural Network Structured Sparse Coding for Online Processing IEEE Access Sparse coding deep neural network weighted iterative shrinkage thresholding algorithm unsupervised learning real-time video denoising |
author_facet |
Haoli Zhao Shuxue Ding Xiang Li Huakun Huang |
author_sort |
Haoli Zhao |
title |
Deep Neural Network Structured Sparse Coding for Online Processing |
title_short |
Deep Neural Network Structured Sparse Coding for Online Processing |
title_full |
Deep Neural Network Structured Sparse Coding for Online Processing |
title_fullStr |
Deep Neural Network Structured Sparse Coding for Online Processing |
title_full_unstemmed |
Deep Neural Network Structured Sparse Coding for Online Processing |
title_sort |
deep neural network structured sparse coding for online processing |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2018-01-01 |
description |
Sparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, has become a mature class of methods with good efficiency in various areas, but it faces limitations in immediate processing such as real-time video denoising. Unsupervised deep neural network structured sparse coding (DNN-SC) algorithms can enhance the efficiency of iterative sparse coding algorithms to achieve the goal. In this paper, we first propose a sparse coding algorithm by adding the idea “weighted" in the iterative shrinkage thresholding algorithm (ISTA), named WISTA, which can enjoy the benefit of the l<sub>p</sub> norm (0 <; p <; 1) sparsity constraint. Then, we propose two novel DNN-SC algorithms by combining deep learning with WISTA and the iterative half thresholding algorithm (IHTA), which is the l<sub>0.5</sub> norm sparse coding algorithm. Furthermore, we present that by changing the loss function, the DNN can be learned supervisedly and unsupervisedly. Unsupervised learning is the key to ensure the DNN to be learned online during processing, which enables the use of the DNN-SC algorithms in applications lacking labels for signals. Synthetic data experiments show that WISTA can outperform ISTA and IHTA. Moreover, the DNNstructured WISTA can successfully achieve converged results of WISTA. In real-world data experiments, the procedure of utilizing DNN-SC algorithms in image denoising is first presented. All DNN-SC algorithms can accelerate at least 45 times while maintaining PSNR results compared with their corresponding sparse coding algorithms. Finally, the strategy of utilizing DNN-SC algorithms in real-time video denoising is presented. The video-denoising experiments show that the DNN-structured ISTA and WISTA can conduct real-time video denoising for 25 frames/s 360 × 480 pixels gray-scaled videos. |
topic |
Sparse coding deep neural network weighted iterative shrinkage thresholding algorithm unsupervised learning real-time video denoising |
url |
https://ieeexplore.ieee.org/document/8542719/ |
work_keys_str_mv |
AT haolizhao deepneuralnetworkstructuredsparsecodingforonlineprocessing AT shuxueding deepneuralnetworkstructuredsparsecodingforonlineprocessing AT xiangli deepneuralnetworkstructuredsparsecodingforonlineprocessing AT huakunhuang deepneuralnetworkstructuredsparsecodingforonlineprocessing |
_version_ |
1724192674528886784 |