An Innovation Study on Applying Deep Learning to Recognize Gesture in Sign Language

• Main Authors: SU, YU-CHEN, 蘇宇楨
• Other Authors: LIAW, HORNG-TWU
• Format: Others
• Language: zh-TW
• Published: 2019
• Online Access: http://ndltd.ncl.edu.tw/handle/6484zw

碩士 === 世新大學 === 資訊管理學研究所(含碩專班) === 108 === Deaf people or dumb people can’t hear or speak, and they usually use sign language to communicate with one another. However, most people are unfamiliar with sign language. Lately, Convolutional Neural Network (CNN) of deep learning has been demonstrated to provide superior performance in image recognition by inputting a large amount of labelled training data. In order to increase the public’s understanding to sign language, this thesis uses deep learning CNN model to recognize Mandarin Phonetic Symbols of sign language which contains 37 different gestures. The first part was to build 720x402 pixels, an average 250 to 300 images of each right-hand shape by smart phone, and there were totally 10,598 images built as a database. The second part was to use these images with 96x96x3 inputting LeNet and SmallVGGNet of CNN model as training, and then adjust the batch size for experiments. The third part was to compare recognition accuracy results of model, and the best training data results are over 96%. Then by testing the model recognition accuracy with different background gesture images, the results get up to 51.35% of LeNet and 29.73% of SmallVGGNet. This thesis built 37 gesture images of Mandarin Phonetic Symbols with single background to train CNN model. The resulting model can be tested with up to 51.35% recognition accuracy with different background gesture images. The experimental process and results can be applied to further studies, and move forward on the language translations between sign and spoken language.