Bad Cost-Effectiveness Tests Undermine…Effectiveness!

Posted by:

Share Button

Or How I Learned to Start Worrying and Hate the TRC

drstangeloveI guess if I were savvier about this, I’d lead in with a Pimsleur-type approach featuring a bikini wearing model to hawk a Spanish lesson. Instead, I’ll dive in with something closer to the matching principal in accounting. That ought to wake you up!

Actually, I’m going to talk about cost-effectiveness tests. Or one test in particular, the TRC.

I confess that when Bob Knight1 first talked with me more than a dozen years ago about the TRC and how it made no sense in a home performance program, I didn’t fully appreciate his argument. That took me a few more years, and it really started to sink in after the first evaluation of NYSERDA’s Home Performance with ENERGY STAR program. Sometimes I’m slow that way. But apparently not as slow as the commissions that still think TRC makes sense.

Whoa, hold on. What is the TRC? It’s the Total Resource Cost Test. And it works something like this. We look at the benefit from an energy efficiency measure, such as insulating, air-sealing, and making some windows changes to a how. And then we compare that to the “total cost” (hence the name) of that energy-saving “resource”, the total cost not being just what the utility paid for both incentives and administration, but also what the homeowner or anyone else paid. The test essentially asks the questions, does this benefit cost (again, based on everything that everyone has spent) less or more than producing that amount of energy.

That sure sounds logical. “We” shouldn’t pay to save energy if we could pay less just to generate it. Remember that sentence. I’ll come back to it.

There is a big problem with that way of thinking, though. In the previous post in this series, I pointed out that people actually buy home improvements for many reasons that have nothing to do with saving energy. They want better light. They want more hot water delivered more quickly. They want their daughter’s bedroom to be warmer in the winter. They want less street noise intruding into a quieter home. And more. All of these things that people spend a lot of money for have a funny name if you look it this through a utility commission lens. They are called “non-energy benefits” or NEBs. And they don’t factor into the TRC because they aren’t energy. TRC looks at all of the costs, but only a small fraction of the benefits. People spend money for those NEBs, but we pretend they spend their money only for energy-savings.

That is nutty. By “nutty” I mean, that doesn’t make sense. The TRC is the wrong test.

Now let me go back to that sentence. “We” shouldn’t pay to save energy if we could pay less just to generate it. (I guess you didn’t have to remember it–I repeated it.) It is fair to say that “ratepayers” shouldn’t pay to save energy if they could pay less just to generate it. Homeowners, however, are free to spend their money on what they want. Cold beer, warm showers, comfortable bedrooms. And they do! They money shouldn’t be weighed against the cost to generate electricity with coal, gas, nukes, or anything else.

Some argue that we can fix the TRC by adjusting how we calculate the benefits, and that we should addthe NEBs into the mix. This means a bunch of people would climb up into an ivory tower and figure out what the value of that cold beer after work is. And how much I value my daughter’s bedroom being warm in the winter and cool in the summer. And how much the “green badge is courage” is worth to me in terms of bragging rights. Of course, the list of potential NEBs is longer than the number of people in the world, so maybe they’ll be stuck up in that tower for a long time. We can only hope!

Another “fix” would be to adjust the cost the homeowner pays, somehow apportioning it without even knowing the real value of the other benefits. Furnace replacement? Let’s assume that happens for free and just count as the cost the difference between a standard replacement in our area and a 95% AFUE model. The good news is that such a methodology could be tweaked to give you just about any answer you want. The bad news is that such a methodology could be tweaked to give you just about any answer you want.

We shouldn’t try to fix the TRC. Instead, we should stop using this as the benchmark.

It’s true, from an energy perspective alone, ratepayers shouldn’t pay to save energy if they could pay less just to generate it.2 I’ll accept that. (I can even accept that in the context of climate change–we can encourage more savings by changing the cost of generating. See my thoughts on the simplest way to get there.) Ratepayers should pay for the energy savings. Only. Now, we can talk about how best to do that, and there are different ways to get there. The best ways involve much better accountability for savings actually delivered and savings that are delivered over time.3 The tests we use, like the utility cost test or program administer cost test, ought to look at that and not how much I value cold beer and warm showers.

When I first drafted this post at the beginning of August as part of this program mistake series, it was much longer, diving more deeply into the arguments. However, since then, this issue has been splashing all over the place, with several good pieces for those who want to learn more. So, you get my slimmed down take, and a list of recommended reading. Good stuff. Be sure to share this with utility commissions, regulators, and staff in your state!

Recommended further reading:

1I’m taking about Bob Knight, retired, but formerly of BKi in Oakland, CA, not Coach Bobby Knight who led the Hoosiers to an NCAA Championship while I was in grad school at Indiana University. This was a few years after his chair-throwing incident, but before he completely lost it and starting decking players. Bob Knight led the charge to knock down the TRC for a long time, including before other started rallying.

2This should be at the portfolio level, not the line item level. Research and development is smart and necessary, and good businesses have to try things that fail to find things that work. That’s how we learn, improve, and get better at things. Those who attack at the line item level without the context of the bigger picture are too clever by half. Or more than half.

3“Delivered over time”. Related to “persistence”. This is something we need to address about purely behavioral approaches. We know that people backslide over time, and we can’t call 3% savings today from behavior changes, 3% savings in perpetuity. Unless we can. But, that 3% might last today only! We’d better be measuring that and proving it, because it doesn’t look like the savings are sticky enough. That is not to knock behavioral approaches. Ultimately, this is all about human choice and behavior! We have to recognize, though, that some actions have longer term consequences than others. If you don’t believe me, ask Anthony Weiner.

<<Previous: Typically, Energy Savings Aren’t the Primary Driver

Next: Workforce training doesn’t create efficiency jobs>>

Share Button
Print Friendly, PDF & Email

About the Author:

Mike Rogers is the President of OmStout Consulting. A nationally recognized expert in residential energy-efficiency, he works with contractors and programs to scale sustainable market approaches to improving homes. More on Google+

Add a Comment