Markov Chain Monte Carlo with Adaptive Proposals

This chapter focuses on different type of adaptive Markov chain Monte Carlo (MCMC) algorithm, for which the proposal distribution can be changed infinitely often during the course of simulation, while preserving stationarity of the target distribution. It then provides an overview of the theory of adaptive MCMC algorithms, and the adaptive Metropolis algorithm and its variants developed under the framework of stochastic approximation. The regeneration time of a Markov chain is a time at which its future becomes independent of the past. Based on this concept, Gilks et al. describe a framework for Markov chain adaptation, which allows the proposal to be modified infinitely often, but preserves the stationarity of the target distribution, and maintains consistency of the sample path averages. The chapter describes how the proposal can be adapted for a Markov chain at regeneration times, using Brockwell and Kadane’s method, and a Metropolis-within-Gibbs procedure to generate new samples. Controlled Vocabulary Terms Gibbs sampling; Markov chain monte carlo; Metropolis-Hastings algorithm; Stochastic approximation; stochastic processes