Bayesian statistics and parameter estimation

Throughout this text, we have considered algorithms to perform simulations – given a model of a system, what is its behavior? We now consider the question of model development. Typically, to develop a model, we postulate a mathematical form, hopefully guided by physical insight, and then perform a number of experiments to determine the choice of parameters that best matches the model behavior to that observed in the set of experiments. This procedure of model proposition and comparison to experiment generally must be repeated iteratively until the model is deemed to be sufficiently reliable for the purpose at hand. The problem of drawing conclusions from data is known as statistical inference , and in particular, our focus here is upon parameter estimation . We use the powerful Bayesian framework for statistics, which provides a coherent approach to statistical inference and a procedure for making optimal decisions in the presence of uncertainty. We build upon the concepts of the last chapter and find, in particular, Monte Carlo simulation to be a powerful and general tool for Bayesian statistics. General problem formulation The basic parameter estimation , or regression , problem involves fitting the parameters of a proposed model to agree with the observed behavior of a system (Figure 8.1). We assume that, in any particular measurement of the system behavior, there is some set of predictor variables x ∈ ℜ M that fully determines the behavior of the system (in the absence of any random noise or error).