Evaluation, Measurement, and Verification (EM&V) for Behavior- Based Energy Efficiency Programs: Issues and Recommendations

Behavior-based programs have been identified as a major potential source of energy savings and are increasingly being adopted by energy efficiency program administrators nationwide. However, because these programs often lack a technology-based unit for which savings can be modeled or deemed (such as a CFL), documentation of energy savings requires approaches which are common in the experimental sciences, but not in the efficiency industry. In this paper (based on a report created for the State and Local Energy Efficiency Action Network), we describe and discuss eleven key evaluation issues and recommend analysis approaches that can be used to define whether a behavior-based program evaluation of energy savings can be considered rigorous and internally valid. In particular, we recommend using randomized controlled trials (RCTs) for behavior-based efficiency programs, which result in robust, unbiased program savings impact estimates. We also discuss issues relating to the double counting of savings, and the importance of third party evaluators. With respect to external validity, we discuss conditions under which impact estimates from behavior-based programs can be applied to different populations in future years. We recommend that a control group that is representative of all of the different participating populations should be maintained for every year in which program energy estimates are being used to claim savings. Finally, we discuss challenges in moving towards estimating savings based on a calibrated predictive analytic model, which could be used to produce deemed savings estimates at some point in the future.