Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets
The statistically equivalent signature (SES) algorithm is a method for feature selection inspired by the principles of constraint-based learning of Bayesian networks. Most of the currently available feature selection methods return only a single subset of features, supposedly the one with the highes...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Foundation for Open Access Statistics
2017-09-01
|
Series: | Journal of Statistical Software |
Subjects: | |
Online Access: | https://www.jstatsoft.org/index.php/jss/article/view/2371 |
id |
doaj-e7c7f70721174755ba7d1178724a5b53 |
---|---|
record_format |
Article |
spelling |
doaj-e7c7f70721174755ba7d1178724a5b532020-11-24T21:21:45ZengFoundation for Open Access StatisticsJournal of Statistical Software1548-76602017-09-0180112510.18637/jss.v080.i071146Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature SubsetsVincenzo LaganiGiorgos AthineouAlessio FarcomeniMichail TsagrisIoannis TsamardinosThe statistically equivalent signature (SES) algorithm is a method for feature selection inspired by the principles of constraint-based learning of Bayesian networks. Most of the currently available feature selection methods return only a single subset of features, supposedly the one with the highest predictive power. We argue that in several domains multiple subsets can achieve close to maximal predictive accuracy, and that arbitrarily providing only one has several drawbacks. The SES method attempts to identify multiple, predictive feature subsets whose performances are statistically equivalent. In that respect the SES algorithm subsumes and extends previous feature selection algorithms, like the max-min parent children algorithm. The SES algorithm is implemented in an homonym function included in the R package MXM, standing for mens ex machina, meaning 'mind from the machine' in Latin. The MXM implementation of SES handles several data analysis tasks, namely classification, regression and survival analysis. In this paper we present the SES algorithm, its implementation, and provide examples of use of the SES function in R. Furthermore, we analyze three publicly available data sets to illustrate the equivalence of the signatures retrieved by SES and to contrast SES against the state-of-the-art feature selection method LASSO. Our results provide initial evidence that the two methods perform comparably well in terms of predictive accuracy and that multiple, equally predictive signatures are actually present in real world data.https://www.jstatsoft.org/index.php/jss/article/view/2371feature selectionconstraint-based algorithmsmultiple predictive signatures |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Vincenzo Lagani Giorgos Athineou Alessio Farcomeni Michail Tsagris Ioannis Tsamardinos |
spellingShingle |
Vincenzo Lagani Giorgos Athineou Alessio Farcomeni Michail Tsagris Ioannis Tsamardinos Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets Journal of Statistical Software feature selection constraint-based algorithms multiple predictive signatures |
author_facet |
Vincenzo Lagani Giorgos Athineou Alessio Farcomeni Michail Tsagris Ioannis Tsamardinos |
author_sort |
Vincenzo Lagani |
title |
Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets |
title_short |
Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets |
title_full |
Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets |
title_fullStr |
Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets |
title_full_unstemmed |
Feature Selection with the R Package MXM: Discovering Statistically Equivalent Feature Subsets |
title_sort |
feature selection with the r package mxm: discovering statistically equivalent feature subsets |
publisher |
Foundation for Open Access Statistics |
series |
Journal of Statistical Software |
issn |
1548-7660 |
publishDate |
2017-09-01 |
description |
The statistically equivalent signature (SES) algorithm is a method for feature selection inspired by the principles of constraint-based learning of Bayesian networks. Most of the currently available feature selection methods return only a single subset of features, supposedly the one with the highest predictive power. We argue that in several domains multiple subsets can achieve close to maximal predictive accuracy, and that arbitrarily providing only one has several drawbacks. The SES method attempts to identify multiple, predictive feature subsets whose performances are statistically equivalent. In that respect the SES algorithm subsumes and extends previous feature selection algorithms, like the max-min parent children algorithm. The SES algorithm is implemented in an homonym function included in the R package MXM, standing for mens ex machina, meaning 'mind from the machine' in Latin. The MXM implementation of SES handles several data analysis tasks, namely classification, regression and survival analysis. In this paper we present the SES algorithm, its implementation, and provide examples of use of the SES function in R. Furthermore, we analyze three publicly available data sets to illustrate the equivalence of the signatures retrieved by SES and to contrast SES against the state-of-the-art feature selection method LASSO. Our results provide initial evidence that the two methods perform comparably well in terms of predictive accuracy and that multiple, equally predictive signatures are actually present in real world data. |
topic |
feature selection constraint-based algorithms multiple predictive signatures |
url |
https://www.jstatsoft.org/index.php/jss/article/view/2371 |
work_keys_str_mv |
AT vincenzolagani featureselectionwiththerpackagemxmdiscoveringstatisticallyequivalentfeaturesubsets AT giorgosathineou featureselectionwiththerpackagemxmdiscoveringstatisticallyequivalentfeaturesubsets AT alessiofarcomeni featureselectionwiththerpackagemxmdiscoveringstatisticallyequivalentfeaturesubsets AT michailtsagris featureselectionwiththerpackagemxmdiscoveringstatisticallyequivalentfeaturesubsets AT ioannistsamardinos featureselectionwiththerpackagemxmdiscoveringstatisticallyequivalentfeaturesubsets |
_version_ |
1725998426824376320 |