The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN

碩士 === 國立交通大學 === 資訊科學與工程研究所 === 106 === This thesis aims to design a hand recognition system based on deep learning CNN with HTC VIVE as the use environment. After a series of analysis, the system achieves a high recognition rate and real-time performance, making the system practical and feasibilit...

Full description

Bibliographic Details
Main Authors: Tsai, Yun-Wei, 蔡運帷
Other Authors: Tsai, Chun-Jen
Format: Others
Language:zh-TW
Published: 2018
Online Access:http://ndltd.ncl.edu.tw/handle/d738ua
id ndltd-TW-106NCTU5394103
record_format oai_dc
spelling ndltd-TW-106NCTU53941032019-05-16T01:00:00Z http://ndltd.ncl.edu.tw/handle/d738ua The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN 頭戴式 VR 的類神經網路手勢辨識系統設計 Tsai, Yun-Wei 蔡運帷 碩士 國立交通大學 資訊科學與工程研究所 106 This thesis aims to design a hand recognition system based on deep learning CNN with HTC VIVE as the use environment. After a series of analysis, the system achieves a high recognition rate and real-time performance, making the system practical and feasibility of the application. In the pre-processing steps, we analyze the usage scenarios of HTC VIVE, including the background removal of hand, the acquisition of position detection, and the depth information. A skin color based method is used in background removal. In terms of hand detection, faster R-CNN based on convolutional neural network model that captures features is used. In addition, considering that the latest HTC VIVE Pro is equipped with a dual-lens camera, we have also tried some stereo matching methods to obtain the depth information. In terms of data collection, we continued to use previous 3D hand models of our laboratory, added other 3D hand models, and collected more real hand data for training and testing. In addition, we also include more angle categories including back and side views, and give a more precise definition of hand gestures to avoid similar hand gesture confusion. In hand recognition, a series of experiments and analysis are performed on the selection of training data, neural network models, and visual analysis. Tsai, Chun-Jen 蔡淳仁 2018 學位論文 ; thesis 72 zh-TW
collection NDLTD
language zh-TW
format Others
sources NDLTD
description 碩士 === 國立交通大學 === 資訊科學與工程研究所 === 106 === This thesis aims to design a hand recognition system based on deep learning CNN with HTC VIVE as the use environment. After a series of analysis, the system achieves a high recognition rate and real-time performance, making the system practical and feasibility of the application. In the pre-processing steps, we analyze the usage scenarios of HTC VIVE, including the background removal of hand, the acquisition of position detection, and the depth information. A skin color based method is used in background removal. In terms of hand detection, faster R-CNN based on convolutional neural network model that captures features is used. In addition, considering that the latest HTC VIVE Pro is equipped with a dual-lens camera, we have also tried some stereo matching methods to obtain the depth information. In terms of data collection, we continued to use previous 3D hand models of our laboratory, added other 3D hand models, and collected more real hand data for training and testing. In addition, we also include more angle categories including back and side views, and give a more precise definition of hand gestures to avoid similar hand gesture confusion. In hand recognition, a series of experiments and analysis are performed on the selection of training data, neural network models, and visual analysis.
author2 Tsai, Chun-Jen
author_facet Tsai, Chun-Jen
Tsai, Yun-Wei
蔡運帷
author Tsai, Yun-Wei
蔡運帷
spellingShingle Tsai, Yun-Wei
蔡運帷
The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
author_sort Tsai, Yun-Wei
title The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
title_short The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
title_full The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
title_fullStr The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
title_full_unstemmed The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN
title_sort design of hand recognition system for head-mounted vr devices using deep learning cnn
publishDate 2018
url http://ndltd.ncl.edu.tw/handle/d738ua
work_keys_str_mv AT tsaiyunwei thedesignofhandrecognitionsystemforheadmountedvrdevicesusingdeeplearningcnn
AT càiyùnwéi thedesignofhandrecognitionsystemforheadmountedvrdevicesusingdeeplearningcnn
AT tsaiyunwei tóudàishìvrdelèishénjīngwǎnglùshǒushìbiànshíxìtǒngshèjì
AT càiyùnwéi tóudàishìvrdelèishénjīngwǎnglùshǒushìbiànshíxìtǒngshèjì
AT tsaiyunwei designofhandrecognitionsystemforheadmountedvrdevicesusingdeeplearningcnn
AT càiyùnwéi designofhandrecognitionsystemforheadmountedvrdevicesusingdeeplearningcnn
_version_ 1719172691647266816