Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors
Automatic identification of human interaction is a challenging task especially in dynamic environments with cluttered backgrounds from video sequences. Advancements in computer vision sensor technologies provide powerful effects in human interaction recognition (HIR) during routine daily life. In th...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2020-07-01
|
Series: | Entropy |
Subjects: | |
Online Access: | https://www.mdpi.com/1099-4300/22/8/817 |
id |
doaj-e4d89548070d4b3b9ad6ab647bcc450c |
---|---|
record_format |
Article |
spelling |
doaj-e4d89548070d4b3b9ad6ab647bcc450c2020-11-25T03:04:12ZengMDPI AGEntropy1099-43002020-07-012281781710.3390/e22080817Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth SensorsAhmad Jalal0Nida Khalid1Kibum Kim2Department of Computer Science, Air University, Islamabad 44000, PakistanDepartment of Computer Science, Air University, Islamabad 44000, PakistanDepartment of Human-Computer Interaction, Hanyang University, Ansan 15588, KoreaAutomatic identification of human interaction is a challenging task especially in dynamic environments with cluttered backgrounds from video sequences. Advancements in computer vision sensor technologies provide powerful effects in human interaction recognition (HIR) during routine daily life. In this paper, we propose a novel features extraction method which incorporates robust entropy optimization and an efficient Maximum Entropy Markov Model (MEMM) for HIR via multiple vision sensors. The main objectives of proposed methodology are: (1) to propose a hybrid of four novel features—i.e., spatio-temporal features, energy-based features, shape based angular and geometric features—and a motion-orthogonal histogram of oriented gradient (MO-HOG); (2) to encode hybrid feature descriptors using a codebook, a Gaussian mixture model (GMM) and fisher encoding; (3) to optimize the encoded feature using a cross entropy optimization function; (4) to apply a MEMM classification algorithm to examine empirical expectations and highest entropy, which measure pattern variances to achieve outperformed HIR accuracy results. Our system is tested over three well-known datasets: SBU Kinect interaction; UoL 3D social activity; UT-interaction datasets. Through wide experimentations, the proposed features extraction algorithm, along with cross entropy optimization, has achieved the average accuracy rate of 91.25% with SBU, 90.4% with UoL and 87.4% with UT-Interaction datasets. The proposed HIR system will be applicable to a wide variety of man–machine interfaces, such as public-place surveillance, future medical applications, virtual reality, fitness exercises and 3D interactive gaming.https://www.mdpi.com/1099-4300/22/8/817cross entropydepth sensorsGaussian mixture modelmaximum entropy Markov model |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Ahmad Jalal Nida Khalid Kibum Kim |
spellingShingle |
Ahmad Jalal Nida Khalid Kibum Kim Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors Entropy cross entropy depth sensors Gaussian mixture model maximum entropy Markov model |
author_facet |
Ahmad Jalal Nida Khalid Kibum Kim |
author_sort |
Ahmad Jalal |
title |
Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors |
title_short |
Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors |
title_full |
Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors |
title_fullStr |
Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors |
title_full_unstemmed |
Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors |
title_sort |
automatic recognition of human interaction via hybrid descriptors and maximum entropy markov model using depth sensors |
publisher |
MDPI AG |
series |
Entropy |
issn |
1099-4300 |
publishDate |
2020-07-01 |
description |
Automatic identification of human interaction is a challenging task especially in dynamic environments with cluttered backgrounds from video sequences. Advancements in computer vision sensor technologies provide powerful effects in human interaction recognition (HIR) during routine daily life. In this paper, we propose a novel features extraction method which incorporates robust entropy optimization and an efficient Maximum Entropy Markov Model (MEMM) for HIR via multiple vision sensors. The main objectives of proposed methodology are: (1) to propose a hybrid of four novel features—i.e., spatio-temporal features, energy-based features, shape based angular and geometric features—and a motion-orthogonal histogram of oriented gradient (MO-HOG); (2) to encode hybrid feature descriptors using a codebook, a Gaussian mixture model (GMM) and fisher encoding; (3) to optimize the encoded feature using a cross entropy optimization function; (4) to apply a MEMM classification algorithm to examine empirical expectations and highest entropy, which measure pattern variances to achieve outperformed HIR accuracy results. Our system is tested over three well-known datasets: SBU Kinect interaction; UoL 3D social activity; UT-interaction datasets. Through wide experimentations, the proposed features extraction algorithm, along with cross entropy optimization, has achieved the average accuracy rate of 91.25% with SBU, 90.4% with UoL and 87.4% with UT-Interaction datasets. The proposed HIR system will be applicable to a wide variety of man–machine interfaces, such as public-place surveillance, future medical applications, virtual reality, fitness exercises and 3D interactive gaming. |
topic |
cross entropy depth sensors Gaussian mixture model maximum entropy Markov model |
url |
https://www.mdpi.com/1099-4300/22/8/817 |
work_keys_str_mv |
AT ahmadjalal automaticrecognitionofhumaninteractionviahybriddescriptorsandmaximumentropymarkovmodelusingdepthsensors AT nidakhalid automaticrecognitionofhumaninteractionviahybriddescriptorsandmaximumentropymarkovmodelusingdepthsensors AT kibumkim automaticrecognitionofhumaninteractionviahybriddescriptorsandmaximumentropymarkovmodelusingdepthsensors |
_version_ |
1724682287897903104 |