Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators
One of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (V...
Main Authors: | , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
PeerJ Inc.
2021-05-01
|
Series: | PeerJ Computer Science |
Subjects: | |
Online Access: | https://peerj.com/articles/cs-429.pdf |
id |
doaj-c94127fe36fe479791ae6c610d159591 |
---|---|
record_format |
Article |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Yuri Antonacci Ludovico Minati Luca Faes Riccardo Pernice Giandomenico Nollo Jlenia Toppi Antonio Pietrabissa Laura Astolfi |
spellingShingle |
Yuri Antonacci Ludovico Minati Luca Faes Riccardo Pernice Giandomenico Nollo Jlenia Toppi Antonio Pietrabissa Laura Astolfi Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators PeerJ Computer Science Granger causality State-space models Vector autoregressive model Artificial neural networks Stochastic gradient descent L1 Multivariate time series analysis |
author_facet |
Yuri Antonacci Ludovico Minati Luca Faes Riccardo Pernice Giandomenico Nollo Jlenia Toppi Antonio Pietrabissa Laura Astolfi |
author_sort |
Yuri Antonacci |
title |
Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators |
title_short |
Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators |
title_full |
Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators |
title_fullStr |
Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators |
title_full_unstemmed |
Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators |
title_sort |
estimation of granger causality through artificial neural networks: applications to physiological systems and chaotic electronic oscillators |
publisher |
PeerJ Inc. |
series |
PeerJ Computer Science |
issn |
2376-5992 |
publishDate |
2021-05-01 |
description |
One of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (VAR) model and on evaluating the information flow between two processes in terms of prediction error variances. In its most advanced setting, GC analysis is performed through a state-space (SS) representation of the VAR model that allows to compute both conditional and unconditional forms of GC by solving only one regression problem. While this problem is typically solved through Ordinary Least Square (OLS) estimation, a viable alternative is to use Artificial Neural Networks (ANNs) implemented in a simple structure with one input and one output layer and trained in a way such that the weights matrix corresponds to the matrix of VAR parameters. In this work, we introduce an ANN combined with SS models for the computation of GC. The ANN is trained through the Stochastic Gradient Descent L1 (SGD-L1) algorithm, and a cumulative penalty inspired from penalized regression is applied to the network weights to encourage sparsity. Simulating networks of coupled Gaussian systems, we show how the combination of ANNs and SGD-L1 allows to mitigate the strong reduction in accuracy of OLS identification in settings of low ratio between number of time series points and of VAR parameters. We also report how the performances in GC estimation are influenced by the number of iterations of gradient descent and by the learning rate used for training the ANN. We recommend using some specific combinations for these parameters to optimize the performance of GC estimation. Then, the performances of ANN and OLS are compared in terms of GC magnitude and statistical significance to highlight the potential of the new approach to reconstruct causal coupling strength and network topology even in challenging conditions of data paucity. The results highlight the importance of of a proper selection of regularization parameter which determines the degree of sparsity in the estimated network. Furthermore, we apply the two approaches to real data scenarios, to study the physiological network of brain and peripheral interactions in humans under different conditions of rest and mental stress, and the effects of the newly emerged concept of remote synchronization on the information exchanged in a ring of electronic oscillators. The results highlight how ANNs provide a mesoscopic description of the information exchanged in networks of multiple interacting physiological systems, preserving the most active causal interactions between cardiovascular, respiratory and brain systems. Moreover, ANNs can reconstruct the flow of directed information in a ring of oscillators whose statistical properties can be related to those of physiological networks. |
topic |
Granger causality State-space models Vector autoregressive model Artificial neural networks Stochastic gradient descent L1 Multivariate time series analysis |
url |
https://peerj.com/articles/cs-429.pdf |
work_keys_str_mv |
AT yuriantonacci estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT ludovicominati estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT lucafaes estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT riccardopernice estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT giandomeniconollo estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT jleniatoppi estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT antoniopietrabissa estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators AT lauraastolfi estimationofgrangercausalitythroughartificialneuralnetworksapplicationstophysiologicalsystemsandchaoticelectronicoscillators |
_version_ |
1721433664799113216 |
spelling |
doaj-c94127fe36fe479791ae6c610d1595912021-05-20T15:05:06ZengPeerJ Inc.PeerJ Computer Science2376-59922021-05-017e42910.7717/peerj-cs.429Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillatorsYuri Antonacci0Ludovico Minati1Luca Faes2Riccardo Pernice3Giandomenico Nollo4Jlenia Toppi5Antonio Pietrabissa6Laura Astolfi7Department of Physics and Chemistry “Emilio Segrè”, University of Palermo, Palermo, ItalyCenter for Mind/Brain Sciences (CIMeC), University of Trento, Trento, ItalyDepartment of Engineering, University of Palermo, Palermo, ItalyDepartment of Engineering, University of Palermo, Palermo, ItalyDepartment of Industrial Engineering, University of Trento, Trento, ItalyIstituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Fondazione Santa Lucia, Rome, ItalyDepartment of Computer, Control and Management Engineering “Antonio Ruberti”, University of Rome “La Sapienza”, Rome, ItalyIstituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Fondazione Santa Lucia, Rome, ItalyOne of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (VAR) model and on evaluating the information flow between two processes in terms of prediction error variances. In its most advanced setting, GC analysis is performed through a state-space (SS) representation of the VAR model that allows to compute both conditional and unconditional forms of GC by solving only one regression problem. While this problem is typically solved through Ordinary Least Square (OLS) estimation, a viable alternative is to use Artificial Neural Networks (ANNs) implemented in a simple structure with one input and one output layer and trained in a way such that the weights matrix corresponds to the matrix of VAR parameters. In this work, we introduce an ANN combined with SS models for the computation of GC. The ANN is trained through the Stochastic Gradient Descent L1 (SGD-L1) algorithm, and a cumulative penalty inspired from penalized regression is applied to the network weights to encourage sparsity. Simulating networks of coupled Gaussian systems, we show how the combination of ANNs and SGD-L1 allows to mitigate the strong reduction in accuracy of OLS identification in settings of low ratio between number of time series points and of VAR parameters. We also report how the performances in GC estimation are influenced by the number of iterations of gradient descent and by the learning rate used for training the ANN. We recommend using some specific combinations for these parameters to optimize the performance of GC estimation. Then, the performances of ANN and OLS are compared in terms of GC magnitude and statistical significance to highlight the potential of the new approach to reconstruct causal coupling strength and network topology even in challenging conditions of data paucity. The results highlight the importance of of a proper selection of regularization parameter which determines the degree of sparsity in the estimated network. Furthermore, we apply the two approaches to real data scenarios, to study the physiological network of brain and peripheral interactions in humans under different conditions of rest and mental stress, and the effects of the newly emerged concept of remote synchronization on the information exchanged in a ring of electronic oscillators. The results highlight how ANNs provide a mesoscopic description of the information exchanged in networks of multiple interacting physiological systems, preserving the most active causal interactions between cardiovascular, respiratory and brain systems. Moreover, ANNs can reconstruct the flow of directed information in a ring of oscillators whose statistical properties can be related to those of physiological networks.https://peerj.com/articles/cs-429.pdfGranger causalityState-space modelsVector autoregressive modelArtificial neural networksStochastic gradient descent L1Multivariate time series analysis |