Bayesian Model Averaging and Bayesian Predictive Information Criterion for Model Selection

The problem of evaluating the goodness of the predictive distributions developed by the Bayesian model averaging approach is investigated. Considering the maximization of the posterior mean of the expected log-likelihood of the predictive distributions (Ando (2007a)), we develop the Bayesian predictive information criterion (BPIC). According to the numerical examples, we show that the posterior mean of the log-likelihood has a positive bias comparing with the posterior mean of the expected log-likelihood, and that the bias estimate of BPIC is close to the true bias. One of the advantages of BPIC is that we can optimize the size of Occam's razor. Monte Carlo simulation results show that the proposed method performs well.