Online Energy Audit – What’s the Use?

Online Energy Audit – What’s the Use?
October 12, 2010 Aaron Goldfeder
Lost Dino

Sure, online energy audit tools are nifty, but do they do anything useful?

Common sense would suggest an online energy audit COULD be useful, but practical experience and caution would suggest that such a tool isn’t actually that useful.  Why?

  • Many online audit tools aren’t easy to use – especially if they require the user to log in or type in utility bills.
  • What if you tell the user the wrong thing?
  • The recommendations are usually cliché – turn off the lights, hang your laundry, etc.
  • Nothing can really replace an actual in-home audit – as experienced contractors know.

So, why bother?

We’ve long believed in the power of an online energy audit in spite of the reasons above. Why?

  • Many homeowners appreciate a place to start and some unbiased information.
  • We believed we could make an experience that people can not only complete but enjoy.
  • If homeowners better understood their potential for savings and recommendations, they’d be more likely to ACTUALLY move to the next step to begin upgrading their homes.

Any luck with that?

We’ve just taken the wraps off of  “EnergySavvy for Programs” – which means the EnergySavvy online energy audit is now available for utility and government energy efficiency programs to embed in their own websites.  And with that, we’re happy to share some of the initial results we’ve seen running the online audit on our site.

Metrics that matter – initial results

There are two metrics we watch carefully with our online energy audit. These are:

Completion Rate – The rate at which homeowners complete the online energy audit.

Conversion Rate – How much more likely users of the online audit are to request an in-home audit or upgrade with a contractor.

The first metric indicates how USABLE the tool is ( i.e. can users actually get through it). The second metric tells you how USEFUL it is in terms of increasing the likelihood for energy saving results.

This means that 80 percent of all visitors who begin the online audit complete it. Also, such users are nearly five times more likely to request a contractor for an in-home energy audit or upgrades than users that do not use the online audit.

Looking at visitors across all segments is only partially useful. In a deeper look across various segments, the results become quite interesting.

From this we see that in some of the key segments, online audit usage is correlated with up to a 10x boost in conversion rate relative to the baseline conversion rate for that segment. That can be a real enhancement in terms of boosting participation.

WARNING: Data geek stuff below

So how did we get these results?

We know that correlation does not imply causation, though we’ve taken pains to make this as rigorous as reasonably possible. The study above was conducted by analyzing a typical set of 75,000 visits to EnergySavvy.com.  The visits were divided into segments based on the visitor’s entry page into the site. Given the entry page, we can roughly assume the visitor’s interest based on the content on that entry point web page.  For example, if a visitor searched Google for “Furnace Rebate” and entered the site on a page about furnace rebates, we add that visitor the “Rebate Interest” segment.

Each segment was then further divided into sub-segments defined by those that had or had not completed the EnergySavvy online energy audit. Each sub-segment was analyzed to see how often that sub-segment requested a contractor for in-home energy efficiency upgrades. This act of requesting a contractor defined the response rate. Each comparison of response rates between sub-segments was checked for statistical significance and all comparisons were found to be of a 99 percent or better confidence level that comparative response rates were not by chance.

A more scientific study would include a full “AB Test” whereby traffic is split into two samples (A and B) and one sample would have the option to do the online energy audit and the other would not.  This would eliminate selection bias whereby it could be the case that the visitors most likely to contact a contractor also happen to be the most likely to want to engage in an online energy audit. Given the dramatic difference in response rates, this is unlikely to nullify these results, but nonetheless an AB test would be an improvement towards a more rigorous result.