EM&V 2.0: Notes from the Field

EM&V 2.0: Notes from the Field
June 5, 2015 Tim Guiterman
Measuring Tape

How Savings Measurement Software Complements Evaluation

Greentech Media published an article this week citing many of the challenges with measuring energy efficiency today, with quotes from me as well as several other voices in the industry. The article addresses very real problems, but it takes a controversial posture without the views of the evaluation community, which inevitably invites some unnecessary conflict. The framing, and especially the headline, implied something I don’t believe: that EM&V as it stands today is inaccurate or unreliable, or that it should go away. I’m not saying that the article misquoted me. And, in fact, I believe that the way we measure energy efficiency should continue to evolve – I said much the same thing in my previous blog post about why I joined EnergySavvy.

While there is a really important conversation here, the take-away message doesn’t reflect EnergySavvy’s views, so I wanted to make sure that a few of our core beliefs are articulated.

Innovation in this field is based on the past and future success of EM&V. More than three decades of successful energy efficiency programs have been quantified and valued as a result of EM&V, and the continued success of this field provides the foundation for new approaches and methods.

New software-based methods won’t replace EM&V work any more than QuickBooks replaced third-party audit firms. EnergySavvy isn’t suggesting that software is going to displace traditional evaluation. Much of the qualitative and quantitative work done by evaluators cannot be automated by software.

Integrated M&V is an exciting and promising frontier. One of the things we’re most excited about is the role of moving savings measurement into the earlier parts of program execution. In an industry that spends over $20B per three-year program cycle, this is an opportunity to create billions of dollars of value through continuous program optimization.

It’s about the customer experience and beyond, not just measurement. The discussion is about more than energy measurement. For example, if on average an installed widget saves 100 kWh, that’s important for planning and incentives. But by definition, some widgets will save less and some more, reflecting quality of installation, customer usage patterns, location, behavior and other factors. Utilities need timely data to identify what works and what doesn’t. And the utility of the future needs to know what’s happening on the grid.

But we cannot ignore the challenges of measurement either. This blog is all about “Notes from the Field,” and since my last post, I have hit the road hard, traveling around the country meeting with utilities, regulators, evaluators, implementers and other stakeholders. And what I’m hearing every day better reflects the challenges and solutions.

There are clear challenges with the way energy efficiency is measured today, with real implications. These generally center around three key areas:

  • First and foremost, data lag is often the biggest source of frustration for regulators and program administrators. The long time delay between program execution and obtaining the final verified savings from impact evaluation means that plans and budgets for the next cycle are set before the results are available. The outcome is that standard operating procedure for many DSM programs is to run a program without a clear understanding of the actual performance.
  • The second challenge builds off the first, and is that the value of measurement directly ties to usefulness. Despite significant spending on EM&V, data lag delays process improvements, threatens program success, and impacts cost effectiveness calculations that determine the regulatory approval of programs. This one is key. The depth and breadth of evaluation studies are substantial, and while I’ve always known there’s some frustration with evaluation, I have been surprised to hear for myself how much of this effort is not perceived as valuable or useful across many departments within utilities. That’s a shame, and something I’d like to see addressed through more than just new software tools.
  • Finally, there is a pent-up demand both at the utilities and in the markets for data based on actual performance. Variability in energy efficiency estimates undercuts the ability to value energy efficiency. It has also been cost-prohibitive to collect information on more than a sample of projects, and this often means that as evaluators, we struggle to provide granular insights due to a lack of sufficient data to dive in deeper. Trade allies and customers also have too few mechanisms to receive feedback on how their energy efficiency projects performed. This gap between expectations and actual performance, whether real or perceived, has direct implications on the confidence that key stakeholders have in energy efficiency.

Technology to help address these challenges is now emerging and in use today. Savings measurement software can provide granular insights into program performance and provide administrators data to improve and enhance programs as well as capture and promote best practices, all during the program year. The utilities I speak with understand how valuable this information is, and for evaluators, continuous and easy-to-digest data helps shine light on the difficult work we do.

Despite these many benefits, this technology is not a silver bullet; savings measurement software works best when we have programs with enough participants and we can tie savings to the meter. Certain programs and applications like custom C&I and retail lighting programs aren’t good candidates. Utilities are also unlikely to invest in software, solely for the sake of M&V, for programs where the engineering estimates of savings are well-established and there is high certainty in the baseline conditions. The challenges that face traditional evaluation methods still remain but using powerful computing and data analytics can bring significant insights, faster, meaning the best solutions will integrate established EM&V expertise, approaches and methods with these new tools. Finally, one more point, and one that I make regularly: savings measurement software is a powerful way to complement and enhance third party EM&V. It will help meet the one key goal of every evaluation, which is to improve the effectiveness of energy efficiency programs, while at the same time leverage deeper value from EM&V efforts.

How we measure and value energy savings is critical to the past and future success of energy efficiency. I look forward to continuing the conversation as I travel to talk with utilities, evaluators, regulators and other stakeholders. EnergySavvy’s next installment on this topic will be a June 18th webinar. I hope you can join us.

-By Tim Guiterman