Accurate Decoding of Imagined and Heard Melodies
Music perception requires the human brain to process a variety of acoustic and music-related properties. Recent research used encoding models to tease apart and study the various cortical contributors to music perception. To do so, such approaches study temporal response functions that summarise the...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2021-08-01
|
Series: | Frontiers in Neuroscience |
Subjects: | |
Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2021.673401/full |
id |
doaj-3bd431dcee4d4a35ab50d4a5055063c7 |
---|---|
record_format |
Article |
spelling |
doaj-3bd431dcee4d4a35ab50d4a5055063c72021-08-05T15:25:42ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2021-08-011510.3389/fnins.2021.673401673401Accurate Decoding of Imagined and Heard MelodiesGiovanni M. Di Liberto0Giovanni M. Di Liberto1Giovanni M. Di Liberto2Giovanni M. Di Liberto3Guilhem Marion4Shihab A. Shamma5Shihab A. Shamma6Laboratoire des Systèmes Perceptifs, CNRS, Paris, FranceEcole Normale Supérieure, PSL University, Paris, FranceDepartment of Mechanical, Manufacturing and Biomedical Engineering, Trinity Centre for Biomedical Engineering, Trinity College, Trinity Institute of Neuroscience, The University of Dublin, Dublin, IrelandCentre for Biomedical Engineering, School of Electrical and Electronic Engineering and UCD University College Dublin, Dublin, IrelandLaboratoire des Systèmes Perceptifs, CNRS, Paris, FranceLaboratoire des Systèmes Perceptifs, CNRS, Paris, FranceInstitute for Systems Research, Electrical and Computer Engineering, University of Maryland, College Park, College Park, MD, United StatesMusic perception requires the human brain to process a variety of acoustic and music-related properties. Recent research used encoding models to tease apart and study the various cortical contributors to music perception. To do so, such approaches study temporal response functions that summarise the neural activity over several minutes of data. Here we tested the possibility of assessing the neural processing of individual musical units (bars) with electroencephalography (EEG). We devised a decoding methodology based on a maximum correlation metric across EEG segments (maxCorr) and used it to decode melodies from EEG based on an experiment where professional musicians listened and imagined four Bach melodies multiple times. We demonstrate here that accurate decoding of melodies in single-subjects and at the level of individual musical units is possible, both from EEG signals recorded during listening and imagination. Furthermore, we find that greater decoding accuracies are measured for the maxCorr method than for an envelope reconstruction approach based on backward temporal response functions (bTRFenv). These results indicate that low-frequency neural signals encode information beyond note timing, especially with respect to low-frequency cortical signals below 1 Hz, which are shown to encode pitch-related information. Along with the theoretical implications of these results, we discuss the potential applications of this decoding methodology in the context of novel brain-computer interface solutions.https://www.frontiersin.org/articles/10.3389/fnins.2021.673401/fullEEGcorticalTRFneural trackingpitchmusic |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Guilhem Marion Shihab A. Shamma Shihab A. Shamma |
spellingShingle |
Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Guilhem Marion Shihab A. Shamma Shihab A. Shamma Accurate Decoding of Imagined and Heard Melodies Frontiers in Neuroscience EEG cortical TRF neural tracking pitch music |
author_facet |
Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Giovanni M. Di Liberto Guilhem Marion Shihab A. Shamma Shihab A. Shamma |
author_sort |
Giovanni M. Di Liberto |
title |
Accurate Decoding of Imagined and Heard Melodies |
title_short |
Accurate Decoding of Imagined and Heard Melodies |
title_full |
Accurate Decoding of Imagined and Heard Melodies |
title_fullStr |
Accurate Decoding of Imagined and Heard Melodies |
title_full_unstemmed |
Accurate Decoding of Imagined and Heard Melodies |
title_sort |
accurate decoding of imagined and heard melodies |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Neuroscience |
issn |
1662-453X |
publishDate |
2021-08-01 |
description |
Music perception requires the human brain to process a variety of acoustic and music-related properties. Recent research used encoding models to tease apart and study the various cortical contributors to music perception. To do so, such approaches study temporal response functions that summarise the neural activity over several minutes of data. Here we tested the possibility of assessing the neural processing of individual musical units (bars) with electroencephalography (EEG). We devised a decoding methodology based on a maximum correlation metric across EEG segments (maxCorr) and used it to decode melodies from EEG based on an experiment where professional musicians listened and imagined four Bach melodies multiple times. We demonstrate here that accurate decoding of melodies in single-subjects and at the level of individual musical units is possible, both from EEG signals recorded during listening and imagination. Furthermore, we find that greater decoding accuracies are measured for the maxCorr method than for an envelope reconstruction approach based on backward temporal response functions (bTRFenv). These results indicate that low-frequency neural signals encode information beyond note timing, especially with respect to low-frequency cortical signals below 1 Hz, which are shown to encode pitch-related information. Along with the theoretical implications of these results, we discuss the potential applications of this decoding methodology in the context of novel brain-computer interface solutions. |
topic |
EEG cortical TRF neural tracking pitch music |
url |
https://www.frontiersin.org/articles/10.3389/fnins.2021.673401/full |
work_keys_str_mv |
AT giovannimdiliberto accuratedecodingofimaginedandheardmelodies AT giovannimdiliberto accuratedecodingofimaginedandheardmelodies AT giovannimdiliberto accuratedecodingofimaginedandheardmelodies AT giovannimdiliberto accuratedecodingofimaginedandheardmelodies AT guilhemmarion accuratedecodingofimaginedandheardmelodies AT shihabashamma accuratedecodingofimaginedandheardmelodies AT shihabashamma accuratedecodingofimaginedandheardmelodies |
_version_ |
1721220270392344576 |