Accounting for selection bias in impact analysis of a rural development program: An application using propensity score matching

When evaluating the impact of a program, the effects of interventions on program outcomes must be measured against a valid counterfactual case. Constructing a valid counterfactual is especially important when experimental data is not available. Building a baseline ensuring that treatment and comparison groups are similar as well as identifying potential sources of bias are essential first steps towards constructing a valid counterfactual. This paper assesses the comparability of groups of participants and non-participants for conducting an impact evaluation of the Agriculture for Basic Needs (A4N) program in Nicaragua. We examine the degree of similarity between A4N participant and non-participant comparison households using propensity scores (estimated probability of program placement). Propensity scores are matched for the two groups, comparing results from using caliper matching and nearest neighbor matching without and with replacement. The analysis uses the pretreatment characteristics of households belonging to the treatment (participant) and comparison (non-participant) groups in order to verify whether the comparison group is statistically similar to the treatment group.