Improvements in SEM Program Impact Evaluation Methods: Lessons Learned from Several Recent Projects

Strategic Energy Management (SEM) programs differ from traditional energy-efficiency programs in that the focus is on integrating a holistic energy-management strategy. These programs provide long-term consulting services that educate and train commercial or industrial energy users to: (1) develop and execute a long-term energy-planning strategy, and (2) integrate energy management into their business planning permanently. Participants implement efficient behaviors, operational and maintenance improvements, and equipment which lead to energy savings. Due to the holistic approach and wide range of activities implemented, the evaluation approach is a whole-building analysis following the IPMVP Option C. Three challenging evaluation areas are discussed: (1) statistically detecting energy savings, (2) designing a sampling strategy, and (3) accounting for equipment and custom measures that received rebates through other programs. Statistically detecting savings is largely dependent upon the length of the data series, whether savings are large enough relative to noise in the energy consumption data, and whether the available data explain the majority of variance in energy consumption. Sampling approaches are not widely employed due to small population sizes and the uniqueness of participating sites, particularly within the industrial sector. Desk reviews, interviews, and site visits can verify equipment and custom measures incented through other programs, each with their own benefits and costs. Lastly, we end with recommendations for evaluators to standardize how results are reported, include confidence intervals around the savings, and better explain how evaluated savings are determined so that other utilities considering SEM programs can review successes and challenges in this area.