Dynamic Gesture Recognition System with Gesture Spotting Based on Self-Organizing Maps

• Main Authors: Hiroomi Hikawa, Yuta Ichikawa, Hidetaka Ito, Yutaka Maeda
• Format: Article
• Language: English
• Published: MDPI AG 2021-02-01
• Series: Applied Sciences
• Subjects: dynamic gesture recognition; gesture spotting; self-organizing map
• Online Access: https://www.mdpi.com/2076-3417/11/4/1933

In this paper, a real-time dynamic hand gesture recognition system with gesture spotting function is proposed. In the proposed system, input video frames are converted to feature vectors, and they are used to form a posture sequence vector that represents the input gesture. Then, gesture identification and gesture spotting are carried out in the self-organizing map (SOM)-Hebb classifier. The gesture spotting function detects the end of the gesture by using the vector distance between the posture sequence vector and the winner neuron’s weight vector. The proposed gesture recognition method was tested by simulation and real-time gesture recognition experiment. Results revealed that the system could recognize nine types of gesture with an accuracy of 96.6%, and it successfully outputted the recognition result at the end of gesture using the spotting result.