Op-Ed: Measuring Energy Efficiency the Right Way

Pranay Kumar, Frank A. Felder | February 19, 2019 | Opinion
New Jersey has invested substantially in measures to improve energy efficiency. The state needs objective, independent, and timely evaluations of these initiatives

Pranay Kumar and Frank Felder
Energy production and consumption account for about two-thirds of the world’s anthropogenic greenhouse-gas (GHG) emissions. Energy efficiency (EE) interventions are not only considered one of the most cost-effective options in cleaning the air and containing climate change, but also relied upon as an effective instrument in saving energy and reducing energy costs.

Despite these claims, it is commonly believed that energy consumers do not utilize efficient products and techniques to their full potential in their daily lives. This disconnect between the theoretically available cost-effective EE potential and the actual realized savings is commonly known as the “energy efficiency gap” or “energy paradox.” The existence, causes, and magnitude of this gap, however, have remained inconclusive despite prolonged debates since the very beginning of the energy conservation measures, initially adopted following the oil crisis in 1970s.

Broadly, these debates have focused on market failures and human behavioral anomalies as possible explanations of the gap. Recent research identifies the importance of modeling and measurement errors in helping to explain the gap. Moreover, the EE gap is dynamic as new technologies enter the market and expand the gap, followed by consumer uptake which reduces it. Consideration of these challenges is important in assessing cost-effectiveness of ratepayer-funded EE programs.

Precise estimation of the energy efficiency gap and cost-effectiveness of EE programs cannot be done meaningfully by following any one analytical frame as the apparent gap has multiple explanations, and the relative contributions of each differ across energy consumers and socio-economic context. Further, the current emphasis on the extent and causes of the EE gap, for its own sake, has to give way to the larger policy goal of reducing carbon emission based on their cost effectiveness. Traditionally, policymakers have relied on active government intervention in the market through subsidies, tax incentives, standards, regulations, and informational campaigns for reducing the EE gap. In fact, the total spending by states in the U.S. utility sector was estimated to be about $7.9 billion in 2017.

Expected savings versus actual savings

Since 2001, New Jersey has invested substantially in incentives provided to over 1.7 million customers that installed EE measures. In 2018, the state passed the Clean Energy Act, which requires utilities to implement EE measures to reduce electricity usage by 2 percent and natural gas usage by 0.75 percent. Earlier, in 2017, New Jersey also enacted a law which allows state taxpayers a tax credit to purchase smart thermostats at up to 50 percent of the cost not exceeding $250. Evaluation of programmable thermostats has revealed actual savings to be significantly short of expected savings.

Impact evaluation of EE programs involves the assessment of EE gaps with reference to a business-as-usual baseline. Estimation of these baselines is considered important not just for the true estimation of economic efficiency, but also from equity considerations between the EE program participants and non-participants. However, it is often difficult to correctly establish the baseline (i.e., the counterfactual) and estimate what would have happened in the absence of specific policy interventions. Peer-reviewed literature on this subject suggests that the actual or net energy savings due to any specific policy or program intervention differ significantly from the gross calculations assuming a flat baseline, citing the concepts of freeridership (FR), spillover (SO), market transformation (MT), and rebound, persistence, and persistence effects. Neglecting these effects in the evaluation of EE programs runs the risk of overstating or understating the benefits significantly.

Another area of concern regarding measurement of the EE gap lies with the large-scale adoption of deemed saving approaches using survey-based methods in the estimation of FR, SO, and other effects. Although these methods are commonly used due to their cost-effectiveness and flexibility, their results are susceptible to potential biases and dependent on the subjective judgment of the evaluator. A more accurate and reliable approach is to estimate net savings using randomized controlled trials, double blind studies, or quasi-experimental methods which control for confounding factors.

Considering the substantial amount of New Jersey ratepayers’ money at stake, it is imperative that the benefits realized from these programs are accurately measured. New Jersey needs objective, independent, and comprehensive monitoring and evaluation of different EE programs on a regular basis to ensure that the overarching objectives of energy conservation in combating climate change are met.

What’s needed

Without objective, independent, and timely evaluations, energy efficiency initiatives will not achieve meaningful results. Any evaluation program should have all the following components:

1. Independent evaluator: An independent evaluator that is organizationally separate from the program administrator should be established. This entity, not the program administrator, should be the organization that determines if, when, and which evaluations should occur. The independent evaluator should procure evaluation studies instead of the program administrator.

2. Long-term and secure funding for the independent evaluator: The independent evaluator should have long-term funding that cannot be reduced, modified or redirected by the program administrator.

3. Complete access to data: The independent evaluator should have access to all data at the same time as the program administrator. It should not have to depend on the program administrator to obtain raw data, analyses, reports, etc.

4. Integration of evaluation with program design and implementation: Evaluation should be integrated into the design and implementation of programs so that data that is generated for program administration can also be used for evaluation purposes. This will reduce the time needed to conduct evaluations and will provide more timely assessments and feedback.

5. Continual evaluation: Evaluation should, where possible, be conducted on a continual basis instead of in a batch mode. The batch model of evaluation — which lets a program run for a year or more and then conducts an evaluation that takes a year or so — does not result in timely and effective evaluations. Program participants should be required to provide data on an ongoing basis that is needed for evaluation purposes even after their initial participation to inform existing and future programs.

6. State-of-the-art evaluation methods: Evaluations should use, where possible, state-of-the-art evaluation methods such as controlled experiments and quasi-controlled experiments.

7. Public reporting: The evaluation entity should be required to make routine, unfettered public reports.

Energy efficiency, if done right, is an important component of any clean-energy effort, and getting it right requires objective, independent and timely evaluations.