Classical Model Selection via Simulated Annealing

SP Brooks, N Friel, Ruth King

Research output: Contribution to journalArticlepeer-review

Abstract

The classical approach to statistical analysis is usually based upon finding values for model parameters that maximize the likelihood function. Model choice in this context is often also based on the likelihood function, but with the addition of a penalty term for the number of parameters. Though models may be compared pairwise by using likelihood ratio tests for example, various criteria such as the Akaike information criterion have been proposed as alternatives when multiple models need to be compared. In practical terms, the classical approach to model selection usually involves maximizing the likelihood function associated with each competing model and then calculating the corresponding criteria value(s). However, when large numbers of models are possible, this quickly becomes infeasible unless a method that simultaneously maximizes over both parameter and model space is available. We propose an extension to the traditional simulated annealing algorithm that allows for moves that not only change parameter values but also move between competing models. This transdimensional simulated annealing algorithm can therefore be used to locate models and parameters that minimize criteria such as the Akaike information criterion, but within a single algorithm, removing the need for large numbers of simulations to be run. We discuss the implementation of the transdimensional simulated annealing algorithm and use simulation studies to examine its performance in realistically complex modelling situations. We illustrate our ideas with a pedagogic example based on the analysis of an autoregressive time series and two more detailed examples: one on variable selection for logistic regression and the other on model selection for the analysis of integrated recapture-recovery data.

Original languageEnglish
Pages (from-to)503-520
Number of pages18
JournalJournal of the Royal Statistical Society: Series B
Volume65
Issue number2
DOIs
Publication statusPublished - May 2003

Keywords

  • autoregressive time series
  • capture-recapture
  • classical statistics
  • information criteria
  • logistic regression
  • Markov chain Monte Carlo methods
  • optimization
  • reversible jump Markov chain Monte Carlo sampling
  • variable selection
  • CONTINUOUS-VARIABLES
  • MAXIMUM-LIKELIHOOD
  • MIXTURE-MODELS
  • OPTIMIZATION
  • ALGORITHM

Fingerprint

Dive into the research topics of 'Classical Model Selection via Simulated Annealing'. Together they form a unique fingerprint.

Cite this