Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models
In this paper, a joint feature selection and parameter estimation algorithm is presented for hidden Markov models (HMMs) and hidden semi-Markov models (HSMMs). New parameters, feature saliencies, are introduced to the model and used to select features that distinguish between states. The feature sal...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2016-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/7450620/ |
id |
doaj-2cb6ad8fbed3410684c25e45a3ee4ae5 |
---|---|
record_format |
Article |
spelling |
doaj-2cb6ad8fbed3410684c25e45a3ee4ae52021-03-29T19:40:36ZengIEEEIEEE Access2169-35362016-01-0141642165710.1109/ACCESS.2016.25524787450620Feature Selection for Hidden Markov Models and Hidden Semi-Markov ModelsStephen Adams0https://orcid.org/0000-0002-1207-4504Peter A. Beling1Randy Cogill2Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA, USADepartment of Systems and Information Engineering, University of Virginia, Charlottesville, VA, USA IBM Ireland Ltd., IBM House, Shelbourne Road, Dublin, IrelandIn this paper, a joint feature selection and parameter estimation algorithm is presented for hidden Markov models (HMMs) and hidden semi-Markov models (HSMMs). New parameters, feature saliencies, are introduced to the model and used to select features that distinguish between states. The feature saliencies represent the probability that a feature is relevant by distinguishing between state-dependent and state-independent distributions. An expectation maximization algorithm is used to calculate maximum a posteriori estimates for model parameters. An exponential prior on the feature saliencies is compared with a beta prior. These priors can be used to include cost in the model estimation and feature selection process. This algorithm is tested against maximum likelihood estimates and a variational Bayesian method. For the HMM, four formulations are compared on a synthetic data set generated by models with known parameters, a tool wear data set, and data collected during a painting process. For the HSMM, two formulations, maximum likelihood and maximum a posteriori, are tested on the latter two data sets, demonstrating that the feature saliency method of feature selection can be extended to semi-Markov processes. The literature on feature selection specifically for HMMs is sparse, and non-existent for HSMMs. This paper fills a gap in the literature concerning simultaneous feature selection and parameter estimation for HMMs using the EM algorithm, and introduces the notion of selecting features with respect to cost for HMMs.https://ieeexplore.ieee.org/document/7450620/Feature selectionhidden Markov modelshidden semi-Markov modelsmaximum a posteriori estimation |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Stephen Adams Peter A. Beling Randy Cogill |
spellingShingle |
Stephen Adams Peter A. Beling Randy Cogill Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models IEEE Access Feature selection hidden Markov models hidden semi-Markov models maximum a posteriori estimation |
author_facet |
Stephen Adams Peter A. Beling Randy Cogill |
author_sort |
Stephen Adams |
title |
Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models |
title_short |
Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models |
title_full |
Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models |
title_fullStr |
Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models |
title_full_unstemmed |
Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models |
title_sort |
feature selection for hidden markov models and hidden semi-markov models |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2016-01-01 |
description |
In this paper, a joint feature selection and parameter estimation algorithm is presented for hidden Markov models (HMMs) and hidden semi-Markov models (HSMMs). New parameters, feature saliencies, are introduced to the model and used to select features that distinguish between states. The feature saliencies represent the probability that a feature is relevant by distinguishing between state-dependent and state-independent distributions. An expectation maximization algorithm is used to calculate maximum a posteriori estimates for model parameters. An exponential prior on the feature saliencies is compared with a beta prior. These priors can be used to include cost in the model estimation and feature selection process. This algorithm is tested against maximum likelihood estimates and a variational Bayesian method. For the HMM, four formulations are compared on a synthetic data set generated by models with known parameters, a tool wear data set, and data collected during a painting process. For the HSMM, two formulations, maximum likelihood and maximum a posteriori, are tested on the latter two data sets, demonstrating that the feature saliency method of feature selection can be extended to semi-Markov processes. The literature on feature selection specifically for HMMs is sparse, and non-existent for HSMMs. This paper fills a gap in the literature concerning simultaneous feature selection and parameter estimation for HMMs using the EM algorithm, and introduces the notion of selecting features with respect to cost for HMMs. |
topic |
Feature selection hidden Markov models hidden semi-Markov models maximum a posteriori estimation |
url |
https://ieeexplore.ieee.org/document/7450620/ |
work_keys_str_mv |
AT stephenadams featureselectionforhiddenmarkovmodelsandhiddensemimarkovmodels AT peterabeling featureselectionforhiddenmarkovmodelsandhiddensemimarkovmodels AT randycogill featureselectionforhiddenmarkovmodelsandhiddensemimarkovmodels |
_version_ |
1724195808515981312 |