3D Depth Information Based 2D Real-Time Hand Gesture Recognition and Skeleton Tracking Technologies

• Main Authors: Ti Chiang, 姜迪
• Other Authors: Chih-Peng Fan
• Format: Others
• Language: zh-TW
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/98620159178186640497

碩士 === 國立中興大學 === 電機工程學系所 === 104 === Owing to the trend of application of virtual reality, wearable devices attract more and more attention recently. Therefore, in this thesis, we propose 2D low-complexity gesture identification technology which is based on 3D depth information and hand skeleton trace. The proposed system can be adapted to two kinds of angles of shooting. As to the distance, the system can tolerate particular range according to different circumstances. In addition to using camera with infrared rays and no colorful information, the system can be used both in daytime and night. The system is processed by using 3D depth information based 2D images, so it can be adapted to two kinds of identification modes. In the proposed real-time gesture recognition system, there are two contexts in applications. The first one is to use head-wearable devices, and the camera can be photographed from the eyes toward the direction of the hand, and then detected palm region for gesture recognition. The second one is to use laptop-like device. The camera can be set up in front of the computer screen, photograph direction towards the user, and it also can detect the palm region for gesture recognition. The proposed system first uses the depth camera to obtain an image of the depth information, and then setting the boundaries according to the needs of different situation. The system can overcome the difficulties for remove complex background, and get a certain distance of the palm. Then the system turn the image of depth of the palm area to plane, and start to search the contour and feature on the contour. After getting the contour of each feature from geometric relationship, the proposed system can recognize common gestures. If the palm recognition generally identifies static or fixed dynamic gesture, features of convex can achieve the purpose. But as a special gesture of static or non-fixed dynamic gesture, it needs more features to assist its judgment. Therefore, it needs to establish streamlined skeleton module. Then, we can track the detailed actions of knuckles. In our experiments, the recognition results are divided into three situations: static gesture: normal-situation identification, static gesture: special-situation identification and dynamic gesture identification, respectively. Among them, static gesture: normal-situation identification can be divided into vertical and parallel forehand and backhand. On the other hand, static gesture: special-situation identification can be divided into bent and close-together status. Furthermore, dynamic gesture identification is partitioned as two common gestures: index finger and two-finger gestures condition. In this work, 4-core PC (Intel® Core™ i7-4720, 2.6GHz, 8GB RAM) is used for implementations, whose average processing speed is up to 15 frames per second.