Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

• Main Authors: Mohammad Farhad Bulbul, Sadiya Tabussum, Hazrat Ali, Wenli Zheng, Mi Young Lee, Amin Ullah
• Format: Article
• Language: English
• Published: MDPI AG 2021-05-01
• Series: Sensors
• Subjects: 3D action recognition; depth motion maps; 3D auto-correlation features; decision fusion; Regularized Collaborative Representation Classifier (CRC)
• Online Access: https://www.mdpi.com/1424-8220/21/11/3642

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>2</mn></msub></semantics></math></inline-formula>-regularized Collaborative Representation Classifier (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>2</mn></msub></semantics></math></inline-formula>-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>2</mn></msub></semantics></math></inline-formula>-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.