All Things M&V 2.0 at the NEEP EM&V Fall Forum

All Things M&V 2.0 at the NEEP EM&V Fall Forum
November 2, 2017 Tim Guiterman
Distributed resources energy analytics

I recently attended the NEEP EM&V Fall Forum, a one-day workshop titled “The Many Flavors of M&V.” The focus of the forum was Advanced Measurement and Verification (AM&V), also called M&V 2.0. The panels covered a variety of topics, including the many uses of “real-time or near-time M&V,” the role of Advanced M&V in evaluation, highlights of recent pilots, and a discussion on guidance and best practices available and needed to best serve the industry.

I joined moderator Carrie Gill (Rhode Island Office of Energy Resources) and fellow panelists Bob Wirtshafter (Wirtshafter and Associates), Emily Cross (Navigant) and Ethan Goldman (VEIC) on the panel focused on the role of Advanced M&V in evaluation. You can read my remarks below. I found the entire day to be an interesting and relevant discussion, and came away with a few observations and takeaways.

First, the EM&V industry has not yet taken a leadership role in adding value to evaluation services by leveraging advances in technology, computing capabilities and increased availability of granular data. Now, before anyone takes this as a blanket statement covering all evaluation firms and providers, let me be clear: the EM&V industry, both across firms and within firms, appears to be split in this regard. There are individuals and groups within some firms that are pushing for new, innovative ways to make M&V and evaluation more timely, targeted and of greater value to the utilities, program implementers, regulators and other stakeholders, including end use customers. However, there are also many that are continuing forward with the status quo, and waiting to react to pressure from regulators or their clients. A lot of the momentum is coming from emerging players in the space, such as technology vendors and analytics companies.

Second, I heard several panelists and attendees refer to a common theme, something akin to “the regulators aren’t asking for…” or “If the regulators want X, then we would provide X.” My reaction to this comment is that most commissioners and commission staff are not islands, and with respect to a field as complex and nuanced as energy efficiency evaluation, they depend on the perspectives, opinions and recommendations of evaluation professionals. For example, nearly all evaluators that I have spoken with agree that there is a problem around the timeliness of insights and results in this field. So if the evaluation community could approach the regulatory bodies in a united way,acknowledge that this is an issue, and point to the various technologies and capabilities, such as Advanced M&V, that can be used to address these issues, they would welcome it.

Third, there is a welcome and growing consensus for the need for standards around measuring energy savings at the meter. The esteemed Miriam Goldberg of DNV GL gave a great presentation on available guidance and protocols, and the dearth of prescriptive standards, that exist in the field of M&V and evaluation today. She correctly noted that as the field of third party energy efficiency evaluation grew in response to the growth in utility-funded efficiency programs, particularly those tied to cost recovery and shareholder incentives, evaluation professionals generally resisted standards, and for understandable reasons. Evaluators want to be able to apply the most applicable measurement approach (e.g IPMVP option A or option C) to the measure or program, and not be constrained from applying professional judgement where necessary. In that context, this makes sense. This is distinct from the discussion of creating sector and application-specific standards to measure savings based on changes in energy usage at customer meters, and accounting for exogenous changes where necessary. These standards, that need to be created with broad stakeholder support – including funding for the expert input – can then be used to provide consistency in the calculation of “metered” energy efficiency – for utilities, regulators, implementers, and private investors. This can allow efficiency to scale well beyond today’s programs, and court more investment from utilities and market actors in the race to procure reliable, low cost and clean resources.

Thanks again to NEEP for hosting a productive discussion on all things related to Advanced M&V, and for the input and wisdom provided by those who attended. At EnergySavvy, we look forward to continued partnerships with the energy efficiency community as we move into an era of “Energy Efficiency 2.0.”

My remarks from the panel follow:

Question prompt: Describe how advanced M&V could play a role in energy efficiency program evaluation.

Advanced M&V (AM&V) can play a role in energy efficiency program evaluation in three key ways:

  • First, by integrating into today’s portfolio evaluations wherever traditional billing analysis – or measurement at the meter – is useful.
  • Second, evaluators can leverage timely feedback to identify areas needed for investigation and strategically inform where to conduct research and allocate EM&V dollars.
  • Third, integration today enables utilities to successfully launch new programs that depend on savings at the meter, and helps evaluators prepare to measure the impacts of EE and other DERs on the grid – essentially, measuring the virtual power plant of the future.

Let me briefly illustrate these points and then get some input from our panel.

Now, AM&V is not explicitly limited to the analysis at the whole building meter level, but for the sake of this question, I’m talking about billing analysis – measurement of energy savings using the whole building utility meter – either monthly or interval, electric or gas. You’ll also hear me use a much more descriptive term for what we at EnergySavvy do – savings measurement software.

So how can AM&V play a role in EE program evaluation?

First, integration. I emphasize this word because I fear that there is still lingering misunderstanding about whether advanced M&V is intended to replace “big E” evaluation. Let me clearly state that no, it is not. And once you remove that misperception, now we’re talking about a measurement approach – an impact analysis approach, which is just one key element in an overall evaluation. And rather than do a billing analysis one time, two years after a program started, we can integrate automated measurement – using a census approach, and in the residential space, with large matched comparison groups of non-participants, and this can be refreshed on any frequency that makes sense for the application – quarterly, monthly, weekly, etc.

So if it makes sense to measure impacts at the meter, then it makes sense to consider using AM&V. But this is not a silver bullet; AM&V works best when we have programs with enough participants and we can tie savings to the meter. Think:

  • Behavioral
  • Connected devices, and controls, and Energy Management Technologies (EMT’s)
  • Whole home or whole building commercial
  • Programs where there are many possible measure combos, including HVAC programs, where deemed savings are not capturing the interactive effects
  • Pilots – where rapid feedback is essential and the pilot is testing complex new technologies – and the savings are not well understood, and certainly not deemed.

Certain programs and applications like upstream rebates and retail lighting programs aren’t good candidates. Utilities or evaluators are also unlikely to invest in software, solely for the sake of impact measurement, for programs where the engineering estimates of savings are well-established and there is high certainty in the baseline conditions.

Now, what does integration look like? We’ve partnered with multiple evaluation firms where the impact analysis for applicable residential programs will use EnergySavvy’s software. So to state very clearly: instead of the traditional firm doing the billing analysis, the teams are proposing to use EnergySavvy’s AM&V tool, and the evaluation firm builds off of that – doing the deeper, more interesting evaluation work – because our clients don’t care about the impact analysis nuts and bolts, they care about the insights gleaned, the red flags raised, the confirmation that savings are real.

Now, we’ve won some of these bids, lost some and have some undecided, so we’re working in this new and evolving market like everyone else, and some utilities might not be ready or willing to try something new. But this is a roadmap for how to integrate it into the evaluation services offered today.

So, what’s new about AM&V that makes this valuable?

This is my second point: savings measurement software builds off of best practices yet incorporates the innovations the software industry is best known for – scalable and portable solutions that utilize big data handling and processing capabilities, automation, machine learning, and dashboard visualizations, to name a few.

That’s where integration leads to something new: A renewed ability to leverage deeper value from EM&V efforts, that can simultaneously address some of the key challenges and deficiencies in current evaluation, and help EM&V improve the effectiveness of energy efficiency programs.

Let me illustrate this point with the integrated evaluation team:

  • In evaluation, one key challenge is identifying areas of uncertainty so we can work to minimize or eliminate it. Currently, we begin evaluations with hints and clues as to what’s working and what’s not – a blank questionnaire. AM&V gives our evaluation team a completely different starting point – a questionnaire that is pre-populated with many of the answers, so we can focus the evaluation efforts and dollars where needed most:
    • These measures, these contractors – no problems here.
    • These customers, business types, measures – appear to be struggling – let’s dig deeper here.
  • Another big challenge: Timeliness. EnergySavvy has been in this space now for four years. Many of our executives are energy professionals but not evaluation professionals. One of the starkest, most salient experiences for them is the willingness of people at all levels – from commission members to individual evaluation managers to say in the very first meeting “it takes too long. We have to make big decisions without knowing the results of our work.” It is in this industry’s best interest to visibly and aggressively address this. Integrating evaluation with AM&V is one way: we can get confident results in the hands of decision makers quicker.
  • A related challenge: lack of timely feedback for implementation and operations. We have seen over and over again that halfway into a program year, AM&V can confidently identify contractors, measures and measure groups, and other characteristics such as demographics and location, etc. that are driving performance. The team is able to share this information with program managers and implementers – in order to help them focus their limited time and resources on what matters most.
  • Another challenge: gathering enough relevant data, specific to unique service territories or customer populations, to inform updates to deemed savings. As you’ll see in one of the pilot presentations (Starting at Slide 39) today, when we integrate automated tools we gather the impacts across thousands of participants and multiple years in one place, and the team has a far richer source of information to inform calibrations or updates to deemed savings values.

My third and final point: the evaluation industry needs to find ways to be relevant in a changing utility landscape.

New program models are not in the future anymore. Now I acknowledge that California is always seen as a one-off, but the programs they are kicking off right now are very illustrative. All four major IOUs have solicitations out for Pay for Performance programs for Energy Management Technologies in the residential and commercial sector. And the measurement will be done using “Normalized Metered Energy Consumption” – aka billing analysis.

The programs need to measure the impacts and base at least some of the incentives off of actual performance. This is a chance for this industry to shine – combining the best of brains and tools to perform continuous, robust accounting of impacts at the customer meter that can support incentive payments and withstand third party scrutiny.

There are also GWh of distributed energy sources all flowing onto the grid that need to be measured and accounted for – and utilities will earn returns on these investments. So if we start by integrating these measurement tools into our portfolio today, we’re ready to apply them to the complex challenges that lay just around the corner. For example:

  • That means understanding EE impacts in terms of time and location. Look at Arizona Public Service (APS’) latest filing -they are proposing programs aimed at “filling the belly of the duck” – we need to be able to accurately measure and report the granular energy and demand impacts in a timely fashion.
  • We also have to understand how EE interacts with DERs.. Eneco – a Dutch utility, is earning a return on what they literally call the “virtual power plant.” Thousands of distributed resources that they need to account for and measure their impact on the nation’s grid.

Now, I care deeply about the future of this industry and the evaluation field itself. I worry that we’ll be relegated to truly to an audit function if we cannot find ways to be proactive and measure the resource. We cannot allow EE measurement to continue to be so time consuming, so complex, and so difficult to communicate the methods and the results that energy efficiency simply gets left behind.