Proper and non-informative conjugate priors for exponential family models

This chapter demonstrates that, in the context of exponential families, under certain conditions, Jeffreys' prior and other non-informative priors — including some forms of ‘unbiased’ priors — can be obtained as suitable limits of conjugate distributions. Moreover, there exists an interesting duality between unbiased estimators and optimal Bayes estimators that minimize expected risk. The chapter is organized as follows. Section 19.2 briefly reviews some basic concepts concerning exponential families and information theory. Section 19.3 discusses Bayesian inference for exponential families based both on proper and certain non-informative, improper conjugate priors. Section 19.4 looks at more general versions of these latter priors and discusses an interesting unbiasedness property of maximum likelihood estimators. Section 19.5 contains some concluding remarks.