The Design of Hand Recognition System for Head-mounted VR Devices using Deep Learning CNN

• Main Authors: Tsai, Yun-Wei, 蔡運帷
• Other Authors: Tsai, Chun-Jen
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/d738ua

碩士 === 國立交通大學 === 資訊科學與工程研究所 === 106 === This thesis aims to design a hand recognition system based on deep learning CNN with HTC VIVE as the use environment. After a series of analysis, the system achieves a high recognition rate and real-time performance, making the system practical and feasibility of the application. In the pre-processing steps, we analyze the usage scenarios of HTC VIVE, including the background removal of hand, the acquisition of position detection, and the depth information. A skin color based method is used in background removal. In terms of hand detection, faster R-CNN based on convolutional neural network model that captures features is used. In addition, considering that the latest HTC VIVE Pro is equipped with a dual-lens camera, we have also tried some stereo matching methods to obtain the depth information. In terms of data collection, we continued to use previous 3D hand models of our laboratory, added other 3D hand models, and collected more real hand data for training and testing. In addition, we also include more angle categories including back and side views, and give a more precise definition of hand gestures to avoid similar hand gesture confusion. In hand recognition, a series of experiments and analysis are performed on the selection of training data, neural network models, and visual analysis.