Propensity score models in observational comparative effectiveness studies: cornerstone of design or statistical afterthought?

Propensity score models are increasingly used in observational comparative effectiveness studies to reduce confounding by covariates that are associated with both a study outcome and treatment choice. Any such potentially confounding covariate will bias estimation of the effect of treatment on the outcome, unless the distribution of that covariate is well-balanced between treatment and control groups. Constructing a subsample of treated and control subjects who are matched on estimated propensity scores is a means of achieving such balance for covariates that are included in the propensity score model. If, during study design, investigators assemble a comprehensive inventory of known and suspected potentially confounding covariates, examination of how well this inventory is covered by the chosen dataset yields an assessment of the extent of bias reduction that is possible by matching on estimated propensity scores. These considerations are explored by examining the designs of three recently published comparative effectiveness studies.