Details for: PG&E Reply to Protest of AL 3935-G_5227-E.pdf

Click on the image for full size preview

Document data

Erik Jacobson
Regulatory Relations

Pacific Gas and Electric Company
77 Beale St., Mail Code B13U
P.O. Box 770000
San Francisco, CA 94177
Fax: 415-973-3582

March 8, 2018

California Public Utilities Commission - Energy Division
Attention: Tariff Unit
505 Van Ness Avenue
San Francisco, CA 94102

Pacific Gas and Electric Company’s Reply to the Protest of Advice
Letter 3935-G/5227-E Request for Authority to Continue the Retail
Products Platform Pilot within PG&E’s Residential Energy Efficiency
Plug-Load and Appliances Sub-Program

Dear Energy Division Tariff Unit:
Pacific Gas and Electric Company (PG&E) hereby replies to a protest dated February
9th, 2018 from Office of Ratepayer Advocates (ORA) to Advice Letter (AL) 3935G/5227-E requesting continued budgetary approval for the third year of the Retail
Products Platform (RPP) pilot within the Statewide Residential Energy Efficiency
program’s Plug-Load and Appliances sub-program. ORA highlights two main areas of
concern: data aggregation and evaluation, and pilot expansion. ORA requests that
PG&E reduce the RPP third year budget proportionate to the expenses and activities
related to the expansion of the program to include dehumidifiers. ORA also requests
the California Public Utilities Commission (‘Commission’ or ‘CPUC’) direct PG&E to
submit a supplemental AL to provide detailed information regarding data aggregation
and evaluation plans. The supplemental AL in question is proposed to include topics
such as additional information on data aggregation issues including a detailed plan and
timeline for addressing them; contingency plans to establish baselines and program
effectiveness if issues are unable to be resolved; and, a realistic schedule for
completing a full and partial evaluation of the pilot including descriptions of the aspects
to be evaluated.
A. Outstanding issues with data aggregation and pilot evaluation
The Commission has requested that PG&E file a supplemental AL that provides further
information on the data aggregation issues, baseline formulation, dehumidifier product
information, and key performance indicators related to PG&E’s RPP logic model. The


- Page 1 -

PG&E’s Reply to Protest of Advice Letter 3935-G/5227-E -2- March 8, 2018 supplemental is being filed on the same day as this reply to protest, March 8th, 2018. Further detail is provided within the AL supplemental. PG&E has worked extensively with the data vendor to make the changes necessary for evaluation, and believes that we will have acceptable data for the evaluators. The Northwest Energy Efficiency Alliance (NEEA) has hired a data vendor to work on data issues outside of the ESRPP portal to complete their early review of the ESRPP program. While NEEA has not yet completed a full evaluation of the program, they feel that working with a consulting firm in addition to the data vendor has given them the data that NEEA needs to move forward with their program evaluation. PG&E would prefer usable and reliable data in the ESRPP portal to complete the early evaluation as originally proposed in the Evaluation Plan, but could consider working with the same consulting firm as NEEA if the known ESRPP portal issues cannot be resolved. The Evaluation Plan1 proposed a theory-driven evaluation and set the objectives for the evaluation as: • Test the hypothesized causal linkages illustrated in the program theory and logic model. • Identify, operationalize, collect, track, and analyze key short and mid-term program performance indicators.2 • Working with the multi-region evaluator, identify and operationalize key long-term market transformation indicators in order to develop baselines that can be tracked over time.3 • Estimate gross and net energy and demand savings. • Continue to test various methods to evaluate the RPP program. • Collaborate with the other Program Administrators and ENERGY STAR in the development and implementation of the multi-region evaluation efforts. While slower than anticipated, significant progress has been made on cleaning the data, and PG&E expects the data will be ready in Q2 2018 to enable our early evaluation consultant to restart the quality assurance/quality control (QA/QC) process. The QA/QC process is the first step in ascertaining whether the portal data is sufficiently clean to allow for early evaluation analyses as outlined in the Evaluation Plan. Assuming the data is sufficient to support a reliable evaluation, we would anticipate the early evaluation taking place Q2 – Q4 2018. To recapitulate, PG&E expects that the following key milestones will take place to address the outstanding data aggregation challenges: Timeframe Q2 2018 Description QA/QC Process: This process will be a thorough review and 1 See “California 2016-2019 Retail Products Platform Program Pilot,” EMI Consulting, 2015, pp. 13-65 2 For more information, see Advice Letter 3668-G/4765-E Appendix A. 3 For more information, see Advice Letter 3668-G/4765-E Appendix A, Table 14.
- Page 2 -

PG&E’s Reply to Protest of Advice Letter 3935-G/5227-E Q2 – Q4 2018 Quarterly (beginning May 2018) -3- March 8, 2018 documentation of portal data to ensure it is being processed in a transparent and reproducible manner and that the dataset is sufficiently robust to allow for meaningful analysis of program impact and attribution. The main elements of the process will include accounting of: a) issues encountered during the QA/QC process and their proposed solutions; b) a data map, guide, and dictionary detailing how a third party could trace all calculations from raw data to relevant outputs; c) measures put in place to check and maintain consistency in the data; and, d) any outstanding issues not resolved along with relevant conclusions and findings. Our ability to include all of these elements is dependent on deliverables from the data vendor and will need to be revisited if any deliverables are not received. Early Evaluation: The approach outlined in the Evaluation Plan might be appropriate; however, the exact method(s) will need to be determined based on data non-participating store sales data from California will be available and we will attempt to use this data in a comparison-based analysis such as difference-in-differences. Given the uncertainty regarding the analytical approach, PG&E will work closely with the CPUC and its consultants to determine the best analytical framework and methodology. PG&E RPP Quarterly Updates: PG&E proposes to hold meetings on a quarterly basis, in which any interested parties may be able to participate. These meetings will provide an update on PG&E RPP program pilot performance, including topics such as timelines, data aggregation and progress on baseline methodologies. PG&E EM&V will resume monthly CPUC-PG&E EM&V meetings once the OA/QC process has been successfully completed. These meetings will be separate from the proposed quarterly meetings. EM&V will work closely with CPUC and its advisors on the early evaluation analysis as done previously in 2016 prior to the early evaluation effort being put on hold. PG&E agrees to provide a quarterly update by the R.13-11-005 service list, as recommended by The Utility Reform Network (TURN) in their supportive response to the pending AL 3935-G/5227-E, to keep stakeholders apprised of the program status. Market transformation programs are complex and have many involved parties. Quarterly updates are a productive means to keep interested parties informed. An update on the early evaluation status and an updated timeline will be a standard agenda item for the quarterly update.
- Page 3 -

PG&E’s Reply to Protest of Advice Letter 3935-G/5227-E -4- March 8, 2018 B. Pilot expansion The expansion to new product categories supports the overall strategy of the RPP pilot as increasing the number of targeted product categories is part of PG&E’s key RPP leading indicators and targets. The original RPP advice letter outlines the annual cumulative number of targeted product categories as part of logic model component. 4 PG&E’s RPP pilot is currently comprised of eight product categories with the original expectation to reach 10 product categories in 2018. Increasing the program scale, including the number of products and program sponsors, will allow a greater influence with retailers. The capability of ESRPP to drive market transformation will rest, in part, on the ability to drive short term behaviors and decisions with incentives. Supporting dehumidifiers requires a relatively minor investment (5% of the incentive budget) from PG&E that in turn supports the national ESRPP effort. In the future, it is likely that PG&E will advocate for other product categories with the national ESRPP program administrator sponsors that are more relevant for California’s climate and market than may be the case in other parts of the country. PG&E respectfully requests that the Commission approve Advice 3935-G-A/5227-E-A as filed. /S/ Erik Jacobson Director, Regulatory Relations cc: Michael Campbell, Program Manager, Office of Ratepayer and 4 Advocates, For more information, see Advice 3668-G/4765-E, Appendix F: PG&E RPP Metrics and Targets, Logic Model Component E2
- Page 4 -