Importance Nested Sampling and the MultiNest Algorithm
Bayesian inference involves two main computational challenges. First, in estimating the parameters of some model for the data, the posterior distribution may well be highly multi-modal: a regime in which the convergence to stationarity of traditional Markov Chain Monte Carlo (MCMC) techniques become...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Maynooth Academic Publishing
2019-11-01
|
Series: | The Open Journal of Astrophysics |
Subjects: | |
Online Access: | https://astro.theoj.org/article/11120-importance-nested-sampling-and-the-multinest-algorithm |
id |
doaj-4a095f848f8640079344c9b5b9ee4ddd |
---|---|
record_format |
Article |
spelling |
doaj-4a095f848f8640079344c9b5b9ee4ddd2020-11-25T03:14:49ZengMaynooth Academic PublishingThe Open Journal of Astrophysics2565-61202019-11-01210.21105/astro.1306.2144Importance Nested Sampling and the MultiNest AlgorithmFarhan Feroz0Michael P. Hobson1Ewan Cameron2Anthony N. Pettitt3University of CambridgeUniversity of CambridgeUniversity of OxfordQueensland University of TechnologyBayesian inference involves two main computational challenges. First, in estimating the parameters of some model for the data, the posterior distribution may well be highly multi-modal: a regime in which the convergence to stationarity of traditional Markov Chain Monte Carlo (MCMC) techniques becomes incredibly slow. Second, in selecting between a set of competing models the necessary estimation of the Bayesian evidence for each is, by definition, a (possibly high-dimensional) integration over the entire parameter space; again this can be a daunting computational task, although new Monte Carlo (MC) integration algorithms offer solutions of ever increasing efficiency. Nested sampling (NS) is one such contemporary MC strategy targeted at calculation of the Bayesian evidence, but which also enables posterior inference as a by-product, thereby allowing simultaneous parameter estimation and model selection. The widely-used MultiNest algorithm presents a particularly efficient implementation of the NS technique for multi-modal posteriors. In this paper we discuss importance nested sampling (INS), an alternative summation of the MultiNest draws, which can calculate the Bayesian evidence at up to an order of magnitude higher accuracy than `vanilla’ NS with no change in the way MultiNest explores the parameter space. This is accomplished by treating as a (pseudo-)importance sample the totality of points collected by MultiNest, including those previously discarded under the constrained likelihood sampling of the NS algorithm. We apply this technique to several challenging test problems and compare the accuracy of Bayesian evidences obtained with INS against those from vanilla NS.https://astro.theoj.org/article/11120-importance-nested-sampling-and-the-multinest-algorithmastronomical data analysisbayesian inferencestatistical methodsimportance nested sampling |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Farhan Feroz Michael P. Hobson Ewan Cameron Anthony N. Pettitt |
spellingShingle |
Farhan Feroz Michael P. Hobson Ewan Cameron Anthony N. Pettitt Importance Nested Sampling and the MultiNest Algorithm The Open Journal of Astrophysics astronomical data analysis bayesian inference statistical methods importance nested sampling |
author_facet |
Farhan Feroz Michael P. Hobson Ewan Cameron Anthony N. Pettitt |
author_sort |
Farhan Feroz |
title |
Importance Nested Sampling and the MultiNest Algorithm |
title_short |
Importance Nested Sampling and the MultiNest Algorithm |
title_full |
Importance Nested Sampling and the MultiNest Algorithm |
title_fullStr |
Importance Nested Sampling and the MultiNest Algorithm |
title_full_unstemmed |
Importance Nested Sampling and the MultiNest Algorithm |
title_sort |
importance nested sampling and the multinest algorithm |
publisher |
Maynooth Academic Publishing |
series |
The Open Journal of Astrophysics |
issn |
2565-6120 |
publishDate |
2019-11-01 |
description |
Bayesian inference involves two main computational challenges. First, in estimating the parameters of some model for the data, the posterior distribution may well be highly multi-modal: a regime in which the convergence to stationarity of traditional Markov Chain Monte Carlo (MCMC) techniques becomes incredibly slow. Second, in selecting between a set of competing models the necessary estimation of the Bayesian evidence for each is, by definition, a (possibly high-dimensional) integration over the entire parameter space; again this can be a daunting computational task, although new Monte Carlo (MC) integration algorithms offer solutions of ever increasing efficiency. Nested sampling (NS) is one such contemporary MC strategy targeted at calculation of the Bayesian evidence, but which also enables posterior inference as a by-product, thereby allowing simultaneous parameter estimation and model selection. The widely-used MultiNest algorithm presents a particularly efficient implementation of the NS technique for multi-modal posteriors. In this paper we discuss importance nested sampling (INS), an alternative summation of the MultiNest draws, which can calculate the Bayesian evidence at up to an order of magnitude higher accuracy than `vanilla’ NS with no change in the way MultiNest explores the parameter space. This is accomplished by treating as a (pseudo-)importance sample the totality of points collected by MultiNest, including those previously discarded under the constrained likelihood sampling of the NS algorithm. We apply this technique to several challenging test problems and compare the accuracy of Bayesian evidences obtained with INS against those from vanilla NS. |
topic |
astronomical data analysis bayesian inference statistical methods importance nested sampling |
url |
https://astro.theoj.org/article/11120-importance-nested-sampling-and-the-multinest-algorithm |
work_keys_str_mv |
AT farhanferoz importancenestedsamplingandthemultinestalgorithm AT michaelphobson importancenestedsamplingandthemultinestalgorithm AT ewancameron importancenestedsamplingandthemultinestalgorithm AT anthonynpettitt importancenestedsamplingandthemultinestalgorithm |
_version_ |
1724642261012054016 |