Multimodal Data Fusion in Learning Analytics: A Systematic Review
Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2020-11-01
|
Series: | Sensors |
Subjects: | |
Online Access: | https://www.mdpi.com/1424-8220/20/23/6856 |
id |
doaj-bdf73755c6a84ed6a9b936562617ebcc |
---|---|
record_format |
Article |
spelling |
doaj-bdf73755c6a84ed6a9b936562617ebcc2020-12-01T00:02:42ZengMDPI AGSensors1424-82202020-11-01206856685610.3390/s20236856Multimodal Data Fusion in Learning Analytics: A Systematic ReviewSu Mu0Meng Cui1Xiaodi Huang2School of Information Technology in Education, South China Normal University, Guangzhou 510631, ChinaSchool of Information Technology in Education, South China Normal University, Guangzhou 510631, ChinaSchool of Computing and Mathematics, Charles Sturt University, Albury NSW 2640, AustraliaMultimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.https://www.mdpi.com/1424-8220/20/23/6856multimodal learning analyticsdata fusionmultimodal datalearning indicatorsonline learning |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Su Mu Meng Cui Xiaodi Huang |
spellingShingle |
Su Mu Meng Cui Xiaodi Huang Multimodal Data Fusion in Learning Analytics: A Systematic Review Sensors multimodal learning analytics data fusion multimodal data learning indicators online learning |
author_facet |
Su Mu Meng Cui Xiaodi Huang |
author_sort |
Su Mu |
title |
Multimodal Data Fusion in Learning Analytics: A Systematic Review |
title_short |
Multimodal Data Fusion in Learning Analytics: A Systematic Review |
title_full |
Multimodal Data Fusion in Learning Analytics: A Systematic Review |
title_fullStr |
Multimodal Data Fusion in Learning Analytics: A Systematic Review |
title_full_unstemmed |
Multimodal Data Fusion in Learning Analytics: A Systematic Review |
title_sort |
multimodal data fusion in learning analytics: a systematic review |
publisher |
MDPI AG |
series |
Sensors |
issn |
1424-8220 |
publishDate |
2020-11-01 |
description |
Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods. |
topic |
multimodal learning analytics data fusion multimodal data learning indicators online learning |
url |
https://www.mdpi.com/1424-8220/20/23/6856 |
work_keys_str_mv |
AT sumu multimodaldatafusioninlearninganalyticsasystematicreview AT mengcui multimodaldatafusioninlearninganalyticsasystematicreview AT xiaodihuang multimodaldatafusioninlearninganalyticsasystematicreview |
_version_ |
1724411282977718272 |