Guided Depth Map Super-Resolution Using Recumbent Y Network
Low spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth infor...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9134754/ |
id |
doaj-2f35f1ff4b494013956700797fd2a2c4 |
---|---|
record_format |
Article |
spelling |
doaj-2f35f1ff4b494013956700797fd2a2c42021-03-30T02:17:53ZengIEEEIEEE Access2169-35362020-01-01812269512270810.1109/ACCESS.2020.30076679134754Guided Depth Map Super-Resolution Using Recumbent Y NetworkTao Li0https://orcid.org/0000-0002-2271-6603Xiucheng Dong1https://orcid.org/0000-0001-6083-7601Hongwei Lin2https://orcid.org/0000-0001-9080-0615School of Electrical Engineering and Electronic Information, Xihua University, Chengdu, ChinaSchool of Electrical Engineering and Electronic Information, Xihua University, Chengdu, ChinaCollege of Electrical Engineering, Northwest Minzu University, Lanzhou, ChinaLow spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth information and intensity information for depth map SR. Specifically, we introduce two weight-shared encoders to respectively learn multi-scale depth and intensity features, and a single decoder to gradually fuse depth information and intensity information for reconstruction. We also design a residual channel attention based atrous spatial pyramid pooling structure to further enrich the feature's scale diversity and exploit the correlations between multi-scale feature channels. Furthermore, the violations of co-occurrence assumption between depth discontinuities and intensity edges will generate texture-transfer and depth-bleeding artifacts. Thus, we propose a spatial attention mechanism to mitigate the artifacts by adaptively learning the spatial relevance between intensity features and depth features and reweighting the intensity features before fusion. Experimental results demonstrate the superiority of the proposed RYNet over several state-of-the-art depth map SR methods.https://ieeexplore.ieee.org/document/9134754/Depth map super-resolutionconvolutional neural networkUNet networkatrous spatial pyramid poolingattention mechanism |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Tao Li Xiucheng Dong Hongwei Lin |
spellingShingle |
Tao Li Xiucheng Dong Hongwei Lin Guided Depth Map Super-Resolution Using Recumbent Y Network IEEE Access Depth map super-resolution convolutional neural network UNet network atrous spatial pyramid pooling attention mechanism |
author_facet |
Tao Li Xiucheng Dong Hongwei Lin |
author_sort |
Tao Li |
title |
Guided Depth Map Super-Resolution Using Recumbent Y Network |
title_short |
Guided Depth Map Super-Resolution Using Recumbent Y Network |
title_full |
Guided Depth Map Super-Resolution Using Recumbent Y Network |
title_fullStr |
Guided Depth Map Super-Resolution Using Recumbent Y Network |
title_full_unstemmed |
Guided Depth Map Super-Resolution Using Recumbent Y Network |
title_sort |
guided depth map super-resolution using recumbent y network |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
Low spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth information and intensity information for depth map SR. Specifically, we introduce two weight-shared encoders to respectively learn multi-scale depth and intensity features, and a single decoder to gradually fuse depth information and intensity information for reconstruction. We also design a residual channel attention based atrous spatial pyramid pooling structure to further enrich the feature's scale diversity and exploit the correlations between multi-scale feature channels. Furthermore, the violations of co-occurrence assumption between depth discontinuities and intensity edges will generate texture-transfer and depth-bleeding artifacts. Thus, we propose a spatial attention mechanism to mitigate the artifacts by adaptively learning the spatial relevance between intensity features and depth features and reweighting the intensity features before fusion. Experimental results demonstrate the superiority of the proposed RYNet over several state-of-the-art depth map SR methods. |
topic |
Depth map super-resolution convolutional neural network UNet network atrous spatial pyramid pooling attention mechanism |
url |
https://ieeexplore.ieee.org/document/9134754/ |
work_keys_str_mv |
AT taoli guideddepthmapsuperresolutionusingrecumbentynetwork AT xiuchengdong guideddepthmapsuperresolutionusingrecumbentynetwork AT hongweilin guideddepthmapsuperresolutionusingrecumbentynetwork |
_version_ |
1724185387035787264 |