Automated Movement Assessment in Stroke Rehabilitation
We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We prop...
Main Authors: | , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2021-08-01
|
Series: | Frontiers in Neurology |
Subjects: | |
Online Access: | https://www.frontiersin.org/articles/10.3389/fneur.2021.720650/full |
id |
doaj-8e5a60f6bbeb4031a7dd650692c37af3 |
---|---|
record_format |
Article |
spelling |
doaj-8e5a60f6bbeb4031a7dd650692c37af32021-08-19T14:49:56ZengFrontiers Media S.A.Frontiers in Neurology1664-22952021-08-011210.3389/fneur.2021.720650720650Automated Movement Assessment in Stroke RehabilitationTamim Ahmed0Kowshik Thopalli1Thanassis Rikakis2Pavan Turaga3Aisling Kelliher4Jia-Bin Huang5Steven L. Wolf6Department of Biomedical Engineering, Virginia Tech, Blacksburg, VA, United StatesGeometric Media Lab, School of Arts, Media and Engineering, Arizona State University, Tempe, AZ, United StatesDepartment of Biomedical Engineering, Virginia Tech, Blacksburg, VA, United StatesGeometric Media Lab, School of Arts, Media and Engineering, Arizona State University, Tempe, AZ, United StatesDepartment of Computer Science, Virginia Tech, Blacksburg, VA, United StatesDepartment of Electrical and Communication Engineering, Virginia Tech, Blacksburg, VA, United StatesDepartment of Rehabilitation Medicine, Emory University, Atlanta, GA, United StatesWe are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).https://www.frontiersin.org/articles/10.3389/fneur.2021.720650/fullstroke rehabilitationautomationcyber-human intelligenceHMMMSTCN++transformer |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Tamim Ahmed Kowshik Thopalli Thanassis Rikakis Pavan Turaga Aisling Kelliher Jia-Bin Huang Steven L. Wolf |
spellingShingle |
Tamim Ahmed Kowshik Thopalli Thanassis Rikakis Pavan Turaga Aisling Kelliher Jia-Bin Huang Steven L. Wolf Automated Movement Assessment in Stroke Rehabilitation Frontiers in Neurology stroke rehabilitation automation cyber-human intelligence HMM MSTCN++ transformer |
author_facet |
Tamim Ahmed Kowshik Thopalli Thanassis Rikakis Pavan Turaga Aisling Kelliher Jia-Bin Huang Steven L. Wolf |
author_sort |
Tamim Ahmed |
title |
Automated Movement Assessment in Stroke Rehabilitation |
title_short |
Automated Movement Assessment in Stroke Rehabilitation |
title_full |
Automated Movement Assessment in Stroke Rehabilitation |
title_fullStr |
Automated Movement Assessment in Stroke Rehabilitation |
title_full_unstemmed |
Automated Movement Assessment in Stroke Rehabilitation |
title_sort |
automated movement assessment in stroke rehabilitation |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Neurology |
issn |
1664-2295 |
publishDate |
2021-08-01 |
description |
We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents). |
topic |
stroke rehabilitation automation cyber-human intelligence HMM MSTCN++ transformer |
url |
https://www.frontiersin.org/articles/10.3389/fneur.2021.720650/full |
work_keys_str_mv |
AT tamimahmed automatedmovementassessmentinstrokerehabilitation AT kowshikthopalli automatedmovementassessmentinstrokerehabilitation AT thanassisrikakis automatedmovementassessmentinstrokerehabilitation AT pavanturaga automatedmovementassessmentinstrokerehabilitation AT aislingkelliher automatedmovementassessmentinstrokerehabilitation AT jiabinhuang automatedmovementassessmentinstrokerehabilitation AT stevenlwolf automatedmovementassessmentinstrokerehabilitation |
_version_ |
1721202227058573312 |