The effects of propensity score weighting on attrition biases in attitudinal, behavioral, and socio-demographic variables in a short-term web-based panel survey

Propensity score weighting often is used to correct for attrition biases in panel surveys. While methodological literature exists on the logic of propensity score weighting and its practical applications, an in-depth discussion is lacking on the effects of using this weighting to correct for attrition biases in attitudinal, behavioral, and socio-demographic variables. By applying a split panel design, we assessed the effects of weighting on attrition biases in 48 attitudinal, 38 behavioral, and 27 socio-demographic variables. Our findings suggest – while acknowledging the estimate-specific nature of bias – first, that biases vary across these types of variables and second, that the effects of propensity score weighting are not homogeneous across the types. Accordingly, a sole reliance on socio-demographics to evaluate attrition in the present panel survey would have resulted in an underestimation of the biases of attitudinal variables and an overestimation of the ability of weighting to cure the negative effects of attrition in behavioral variables.