SEPTEMBER FILINGS
Contact
Shawn-Dai Linderman ([email protected])
1. Please provide a summary of your organization’s general comments on the Revised Straw Proposal (RSP) and the meeting and materials shared on September 11th.
The California Community Choice Association (CalCCA) appreciates the opportunity to provide comments on the California Independent System Operator’s (CAISO’s) RSP and the September 11, 2024, stakeholder meeting. CalCCA supports the CAISO's revised schedule, recognition of this effort as an interim solution, and commitment to conducting a subsequent effort to more comprehensively review storage bid cost recovery (BCR) issues. CalCCA supports the direction of the RSP, which allows for the exploration of the CAISO’s proposed solution and alternatives presented by parties. More holistic solutions are likely necessary to ensure storage resources can fully represent real-time conditions in their bids.
2. Please provide your organization’s comments regarding the changes on the RSP relative to the Issue Paper & Straw Proposal (IPSP).
CalCCA supports the direction of the CAISO’s RSP. The RSP does not advance a single proposed solution.
Instead, it recognizes the impacts the multi-interval optimization (MIO) and market power mitigation (MPM) can have on the dispatch of storage and puts forth multiple options for stakeholders to consider, including the CAISO’s proposed solution and alternative solutions put forth by stakeholders. The CAISO has legitimate concerns that (1) storage assets are not exposed to real-time prices for deviating from day-ahead schedules, and (2) storage assets are incentivized to bid strategically to maximize the combined BCR and market payment. [1] The CAISO’s proposed solution and the alternative solutions should be evaluated based upon how well they can address these concerns in the interim. More holistic solutions are likely necessary to fully resolve these concerns, however, by ensuring storage resources can fully represent real-time conditions in their bids.
The CAISO’s proposed solution to redefine dispatch that is unavailable due to state-of-charge (SOC) constraints as ineligible for BCR has the “fundamental assumption” that storage dispatch is optimal for the binding interval, which is not always the case under the MIO. The result, the CAISO explains, is that the Proposed Solution would not significantly reduce BCR payments and would be challenging to implement. Additionally, CalCCA is still concerned that the CAISO’s proposal does not explicitly target instances when BCR is unwarranted.[2] For these reasons, CalCCA supports the parties with alternative proposals working together to identify areas of alignment among their alternatives, which appear numerous, and areas of unalignment (e.g., whether to apply the alternative calculations in all intervals or only the binding interval and whether to use default energy bid (DEB) values in the calculation) so that the CAISO and parties can work towards building a consensus proposal. [1] RSP at 9.
[2] CalCCA Comments on Alternative Proposals (Aug. 26,
2024): https://stakeholdercenter.caiso.com/Comments/AllComments/6a3c5d4b-3018-4a56-91bb-9d0b341caac2#org-adb2662f-ca44-469e-9452-b6a7a59b7f95.
3. Please provide your organization's comments regarding the ISO Proposed Solution and the relevant discussion in sections 5 and 6 of the RSP.
See response in Section 2.
California Community Choice Association SUBMITTED 09/23/2024, 02:06 PM 10/3/24, 6:21 PM California ISO - All comments
https://stakeholdercenter.caiso.com/Comments/AllComments/04A134B1-3874-498A-AB08-E075048095E5#org-549cb435-793f-4750-ac49-2f10d54e4… 1/2
4. Please provide your organization's comments regarding the CESA Alternative Solution, its variations, and the relevant discussion in sections 5 and 6 of the RSP.
See response in Section 2.
5. Please provide your organization's comments regarding the Vistra Alternative Solution and the relevant discussion in sections 5 and 6 of the RSP.
See response in Section 2.
6. Please provide your organization's comments regarding the issues related to local market power mitigation (LMPM) as described in the RSP and in the September 11th materials.
CalCCA appreciates the analysis provided by DMM related to LMPM and its impacts on storage bids and dispatches. The data revealed that the increase in storage dispatch due to mitigation is small, especially in the hours prior to the net load peak hours. BCR could be warranted in these cases, even if infrequent, and the CAISO’s solution should consider these cases. Until the CAISO can develop improved storage DEBs that better reflect potential opportunity costs of dispatching storage in real-time in advance of day-ahead schedules, the CAISO should consider instances of mitigation when determining if BCR is warranted. CalCCA agrees with CAISO that the way the CAISO considers mitigation “will be largely dependent on further analysis and the solution pursued.”[1]
[1] RSP at 21.
7. Please provide your organization's comments regarding the issues related to applicable intervals and multi-interval optimization (MIO) as described in the RSP and in the September 11th materials.
CalCCA appreciates the examples presented by CAISO explaining the issues for storage BCR related to the MIO and the analysis presented by DMM demonstrating the drivers of real-time BCR from SOC induced buy/sell backs. DMMs’ analysis demonstrates that BCR for battery storage with SOC induced buy/sell backs is primarily driven by negative revenues, not the resources’ bid costs. This analysis highlights the importance of ensuring the solution recognizes when BCR is warranted or unwarranted. For example, if the CAISO market charges a storage resource out-of-market based on advisory prices that do not materialize later in the day and this results in negative revenues, the MIO could be driving warranted BCR.
8. Please provide any other proposals or variations of a proposal that your organization considers the ISO should assess. Please include as much detail as possible.
CalCCA has no other proposals at this time.
9. Please provide your organization’s comments regarding the Governance Classification.
CalCCA supports this initiative’s Joint Authority Governance Classification.
10. Please provide any additional comments, feedback, or examples regarding the stakeholder meeting. You may upload examples or data using the “Attachments” field below.
CalCCA has no additional comments at this time.
10/3/24, 6:21 PM California ISO - All comments
https://stakeholdercenter.caiso.com/Comments/AllComments/04A134B1-3874-498A-AB08-E075048095E5#org-549cb435-793f-4750-ac49-2f10d54e4… 2/2 Advice 20-E /20-G October 1, 2024
1 October 1, 2024
Advice Letter 20-E/20-G (Southern California Regional Energy Network–CPUC ID #940) Advice Letter 27-E (Bay Area Regional Energy Network–CPUC ID #941) Advice Letter 6-E/6-G (Inland Regional Energy Network–CPUC ID #246) Advice Letter 83-E (Marin Clean Energy)
Advice Letter 4980-G/7390-E (Pacific Gas & Electric Company–CPUC ID # U 39 E) Advice Letter 4523-E/3351-G (San Diego Gas & Electric Company–CPUC ID # U 902 E) Advice Letter 5383-E (Southern California Edison Company–CPUC ID #U 338 E) Advice Letter 6375-G (Southern California Gas Company–CPUC ID #U 904 G) Advice Letter 12-E/11-G (Tri-County Regional Energy Network–CPUC ID #220) Public Utilities Commission of the State of California
SUBJECT: SOUTHERN CALIFORNIA REGIONAL ENERGY NETWORK FILING ON
BEHALF OF THE ENERGY EFFICIENCY PORTFOLIO ADMINISTRATORS FOR
PROGRAM OVERLAP ANALYSIS
I. PURPOSE
Pursuant to Ordering Paragraph (OP) 32 of Decision (D.) 23-06-055, the Southern California Regional Energy Network (SoCalREN), hereby submits for approval with the California Public Utilities Commission (CPUC or Commission) this Tier 2 Advice Letter on behalf of the California energy efficiency (EE) Portfolio Administrators (PAs)1, herein after referred to as the “Joint PAs”.
On August 16, 2024, SoCalREN, on behalf of the Joint PAs, requested a 30-day extension to October 1, 2024, to comply with OP 32. On August 21, 2024, the Executive Director of Energy 1 The PAs include Bay Area Regional Energy Network (BayREN), Inland Regional Energy Network (I-REN), Marin Clean Energy (MCE), Pacific Gas and Electric Company (PG&E), San Diego Gas & Electric Company (SDG&E), Southern California Edison Company (SCE), Southern California Gas Company (SoCalGas), Southern California Regional Energy Network (SoCalREN), and Tri-County Regional Energy Network (3C-REN).
County of Los Angeles
INTERNAL SERVICES DEPARTMENT
1100 North Eastern Avenue
Los Angeles, California 90063
Speed. Reliability. Value.
Telephone: (323) 267-2101
FAX: (323) 264-7135
Advice 20-E /20-G October 1, 2024
2 Division granted the extension. As such, the Joint PAs request approval of the information provided in the attached report (OP32 Report).
Specifically, OP 32 ordered:
By no later than September 1, 2024, the portfolio administrators (PAs) shall submit a joint Tier 2 advice letter to provide information on substantively similar programs, and steps they have taken and will take to mitigate or minimize ratepayer risk of program overlap and duplication. The joint advice letter must provide:
a) A comprehensive list of any substantively similar ratepayer-funded energy efficiency programs among the PAs.
b) A clear statement of the issues or problems that result from program offerings identified in item (a).
c) Definitions or clarifications of any jargon that PAs suggest specifying, in order to have a shared understanding of the issue or problems associated with substantively similar ratepayer-funded energy efficiency programs among different PAs. For example, the joint advice letter should propose definitions for “overlap,” “duplication,” and “precedence.” d) Description of the risk to ratepayers of ratepayer-funded energy efficiency programs that “overlap” or are otherwise “duplicative” (as defined in response to Item (c), above) in some substantive way. What is the estimated dollar value of the risk? Describe how the value was calculated or assessed.
e) Description of the actions, measures, etc. that PAs have taken thus far to identify and mitigate or minimize risks to ratepayers of substantively similar ratepayer-funded energy efficiency programs among different PAs, and any other issues identified in response to Item (b).
Include, for instance, any agreed-upon criteria for determining program “overlap,” “duplication,” and/or “precedence,” and what steps have been taken by each PA in cases where “overlap” or “duplication” was identified. Explain the effectiveness of each of these measures, actions, etc. in mitigating ratepayer risks, and identify and describe what issues remain unresolved. f) Description of how the PAs will effectively mitigate or minimize ratepayer risks associated with similar ratepayer-funded energy efficiency programs among different PAs through the joint cooperation memoranda or any other agreed-upon process or protocol.
II. BACKGROUND
In D.23-06-055, the Commission states it is:
not wedded to establishing formal guidance if the PAs demonstrate they have a process or protocol to, among themselves, effectively mitigate or minimize ratepayer risks associated with duplicative or overlapping programs. Therefore, we will direct the PAs to submit a joint Tier 2 advice letter to provide information on substantively similar programs, and steps they have taken and will take to mitigate or minimize ratepayer risk of program overlap or duplication.2
2 D.23-06-055 at 87.
Advice 20-E /20-G October 1, 2024
3 To comply with this directive and OP 32, the Joint PAs designated SoCalREN and SDG&E as the co-lead PAs to coordinate the development of the information required by OP 32, with input from and agreement by all the PAs. In consultation with the Joint PAs, SoCalREN hired a consultant to help facilitate discussions and the development of the required data into a report. SoCalREN was also designated as the lead PA to submit the Tier 2 Advice Letter. The Joint PAs’ responses to OP 32 were compiled into a report (OP32 Report, attached hereto as Attachment 1). Given the length of the report, and the multiple sub-objectives of OP 32, the OP32 Report contains a guidance table that references where the response to each sub-objective is located.
Joint Statement from PG&E, SCE, and SDG&E
The Joint Advice Letter and Final Report comply with direction in OP 32 and highlight the ability for substantively similar or duplicative programs to coexist across EE PA portfolios if proper controls are in place. The Joint PAs plan to continue further analysis to comprehensively capture the scope of program overlap risk. Areas warranting further analysis include measure-level assessments, where detailed insights can identify specific points of overlap, including among multiple programs across different market intervention channels.3 In addition, deliberate program overlap, if strategically coordinated, can offer benefits to ratepayers by enhancing access to various solutions while minimizing administrative redundancies and expanding the reach of ratepayer funds to complementary programs. Finally, given that program precedence determinations may need to take in complex considerations on ratepayer risk, it is appropriate for the Commission to continue leveraging the Joint Cooperation Memorandum (JCMs) and PA-to-PA coordination efforts, where the necessary expertise and flexibility exist, unless the PAs determine that an alternative pathway for establishing precedence is necessary after determining the full scope of program overlap risk.
3 Different market intervention channels include upstream/midstream and downstream delivery channels, statewide and local programs, and between statewide programs.
Advice 20-E /20-G October 1, 2024
4 III. PROTESTS
Anyone may protest this advice letter. The protest must state the grounds upon which it is based.
The protest must be made in electronic form and received by the Commission within 20 days of the date this advice letter was submitted. There is no restriction on who may file a protest.
Protests must be submitted to:
Public Utilities Commission
CPUC Energy Division
Attention: Tariff Unit
Email: [email protected]
It is also requested that a copy of the protest be sent by email to addresses shown below on the same date it is delivered to the Commission.
For 3C-REN:
Alejandra Tellez
Deputy Executive Officer
Ventura County
800 S. Victoria Avenue
Ventura, CA 93009
805-654-3835
Email: [email protected]
For BayREN:
Jane Elias
Section Director, Energy Programs
Association of Bay Area Governments
375 Beale Street, Suite 700
San Francisco, CA 94105
Email: [email protected]
For I-REN:
Casey Dailey, Director of Energy & Environmental Programs Western Riverside Council of Governments
3390 University Ave., Suite 200
Riverside, California 92501
(951) 405-6720
Email: [email protected]
For MCE:
Wade Stano
Senior Policy Counsel
Advice 20-E /20-G October 1, 2024
5 Email: [email protected]
For PG&E:
Sidney Bob Dietz II
Director, Regulatory Relations
c/o Megan Lawson
Email: [email protected]
For SCE:
Connor Flanigan
Managing Director, State Regulatory Operations
Email: [email protected]
Adam Smith
Director, Regulatory Relations
c/o Karyn Gansecki
Email: [email protected]
For SDG&E:
Attn: Greg Anderson
Regulatory Tariff Manager
Email: [email protected]
Email: [email protected]
For SoCalGas:
Attn: Gary Lenart
Regulatory Tariff Manager
Email: [email protected]
Email: [email protected]
For SoCalREN:
Lujuana Medina, Division Manager
Environmental Initiatives Division
County of Los Angeles
1100 North Eastern Avenue
Los Angeles, CA 900633200
(323) 391-8537
Email: [email protected]
Advice 20-E /20-G October 1, 2024
6 IV. EFFECTIVE DATE
Pursuant to D.23-06-055, this advice letter is classified as Tier 2 and is subject to Commission Staff approval. Protests are pursuant to General Order (GO) 96-B, Section 7.4. SoCalREN respectfully requests that Commission Staff expeditiously review and approve this advice letter.
V. NOTICE
A copy of this Advice Letter is being sent to the Commission’s service lists for R.13-11-005 and A.22-02-005 (et al.). For changes to R.13-11-005 or A.22-02-005 (et al.) service lists, please contact the Commission’s Process Office at 415-703-2021 or by electronic mail at [email protected].
Respectfully Submitted,
/s/ Lujuana Medina
Lujuana Medina, Division Manager
Environmental Initiatives
Energy and Environmental Services
County of Los Angeles Office
1100 North Eastern Avenue
Los Angeles, CA 900633200
(323) 391-8537
[email protected]
California Public Utilities Commission
MUST BE COMPLETED BY UTILITY (Attach additional pages as needed) Company name/CPUC Utility No.: ::,outhern Lalltorma Keg10nal Energy Network l'+'J4U Utility type: Contact Person: Lu1uana Medlna
ELC GAS WATER Phone#: (jLj) j';ll-1S:>j /
E-mail: [email protected]
PLC HEAT E-mail Disposition Notice to: [email protected] EXPLANATION OF UTILITY TYPE (Date Submitted / Received Stamp by CPUC) ELC = Electric GAS= Gas w ATER = Water PLC = Pipeline HEAT= Heat Advice Letter (AL) #: 2U-E/2U-G, et al. Tier Designation: 2 Subject of AL: SOUTHERN CALIFORNIA REGIONAL ENERGY NETWORK FILING ON BEHALF OF THE
ENERGY EFFICIENCY PORTFOLIO ADMINISTRATORS FOR PROGRAM OVERLAP
ANALYSIS
Keywords (choose from CPUC listing):
AL Type: D Monthly D Quarterly D Annual 0 One-Time D Other:
If AL submitted in compliance with a Commission order, indicate relevant Decision/Resolution #:
Decision (D.123-06-055
Does AL replace a withdrawn or rejected AL? If so, identify the prior AL: No Summarize differences between the AL and the prior withdrawn or rejected AL: N / A Confidential treatment requested? D Yes No
If yes, specification of confidential information:
Confidential information will be made available to appropriate parties who execute a nondisclosure agreement. Name and contact information to request nondisclosure agreement/ access to confidential information:
Resolution required? Yes D No
Requested effective date: 11 / 1/ 24
Estimated system annual revenue effect (%): N; A
Estimated system average rate effect (%) : N; A
No. of tariff sheets: N/ A
When rates are affected by AL, include attachment in AL showing average rate effects on customer classes (residential, small commercial, large C/1, agricultural, lighting).
Tariff schedules affected: N/ A
Service affected and changes proposed 1: N/ A
Pending advice letters that revise the same tariff sheets: N/ A I more space is Clear Form
California Public Utilities Commission
Energy Division Tariff Unit Email:
[email protected]
Protests and correspondence regarding this AL are to be sent via email and are due no later than 20 days after the date of this submittal, unless otherwise authorized by the Commission, and shall be sent to:
Contact Name:
Title:
Utility/Entity Name:
Telephone (xxx) xxx-xxxx:
Facsimile (xxx) xxx-xxxx:
Email:
Contact Name:
Title:
Utility/Entity Name:
Telephone (xxx) xxx-xxxx:
Facsimile (xxx) xxx-xxxx:
Email:
CPUC
Energy Division Tariff Unit
505 Van Ness Avenue
San Francisco, CA 94102
Division Manager
County of Los Angeles Office
Lujuana Medina
(323) 3918537
[email protected]
Clear Form
ENERGY Advice Letter Keywords
Affiliate Direct Access Preliminary Statement
Agreements Disconnect Service Procurement
Agriculture ECAC / Energy Cost Adjustment Qualifying Facility Avoided Cost EOR / Enhanced Oil Recovery Rebates
Balancing Account Energy Charge Refunds
Baseline Energy Efficiency Reliability
Bilingual Establish Service Re-MAT/Bio-MAT
Billings Expand Service Area Revenue Allocation
Bioenergy Forms Rule 21
Brokerage Fees Franchise Fee / User Tax Rules
CARE G.O. 131-D Section 851
CPUC Reimbursement Fee GRC / General Rate Case Self Generation Capacity Hazardous Waste Service Area Map
Cogeneration Increase Rates Service Outage
Compliance Interruptible Service Solar
Conditions of Service Interutility Transportation Standby Service Connection LIEE / Low-Income Energy Efficiency Storage
Conservation LIRA / Low-Income Ratepayer Assistance Street Lights Consolidate Tariffs Late Payment Charge Surcharges
Contracts Line Extensions Tariffs
Core Memorandum Account Taxes
Credit Metered Energy Efficiency Text Changes
Curtailable Service Metering Transformer
Customer Charge Mobile Home Parks Transition Cost
Customer Owned Generation Name Change Transmission Lines Decrease Rates Non-Core Transportation Electrification
Demand Charge Non-firm Service Contracts Transportation Rates Demand Side Fund Nuclear Undergrounding
Demand Side Management Oil Pipelines Voltage Discount
Demand Side Response PBR / Performance Based Ratemaking Wind Power Deposits Portfolio Withdrawal of Service
Depreciation Power Lines
Clear Form
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
1 Attachment 1
D.23-06-055 Ordering Paragraph 32
Portfolio Administrators Report on Ratepayer-Funded Energy Efficiency Programs (OP32 Report)
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
2 Contents
Introduction and Overview ........................................................................................................................ 6 Background Information ........................................................................................................................... 7 Decision Requirements ......................................................................................................................... 7 Compliance with Decision Guidance .................................................................................................... 8 Background Context and Document Review ........................................................................................ 9 Summary of Findings .............................................................................................................................. 10 Project Approach ...................................................................................................................................... 11 Establish Steering Committee ................................................................................................................. 13 Identify and Define Relevant Terms and Criteria ................................................................................... 13 Development of Program Matrix Tool V1 ............................................................................................... 14 Explore Programs by Sector for Overlap................................................................................................ 14 Development of Program Matrix Tool V2 ............................................................................................... 17 Explore, Qualify, and Quantify Program Overlap Risk by PA ................................................................ 18 Aggregate and Analyze Data .................................................................................................................. 18 Produce, Review, and Finalize Report and Advice Letter ....................................................................... 19 Definitions .................................................................................................................................................. 19 Similar (Level 1) ..................................................................................................................................... 19 Program Overlap (Level 2) ..................................................................................................................... 19 Substantively Similar (Level 3) ............................................................................................................... 20 Program Duplication (Level 4) ............................................................................................................... 20 Methodology .............................................................................................................................................. 22 Program Duplication Spectrum Scoring ................................................................................................. 23 Mitigation Options and Scoring .............................................................................................................. 28 Ratepayer Risk ........................................................................................................................................ 29 Findings and Analysis ............................................................................................................................... 32 D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
3 Summary of Program Administrator Responses ..................................................................................... 32 Overlap by PA ......................................................................................................................................... 34 Mitigation Strategies ............................................................................................................................... 35 Summary of Mitigation Strategies Identified ...................................................................................... 35 Ratepayer Risk ........................................................................................................................................ 37 Additional Analysis by Segment .............................................................................................................. 39 Additional Analysis by Sector ................................................................................................................. 40 Issues Resulting from Analysis and Unresolved Issues .......................................................................... 43 Issues Resulting from Analysis ................................................................................................................ 43 Unresolved Issues ................................................................................................................................... 43 Next Steps .................................................................................................................................................. 46 Appendix A ................................................................................................................................................ 48 Appendix B ................................................................................................................................................ 49
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
4 Index of Tables
Table 1. Acronyms Used in Report ............................................................................................................... 5 Table 2. OP32 Assignment Compliance Table .............................................................................................. 8 Table 3. Project Approach Summary .......................................................................................................... 11 Table 4. Activities to Identify and Define Relevant Terms ......................................................................... 13 Table 5. Activities to Explore Programs by Sector for Overlap .................................................................. 14 Table 6. Overview of Sector-Specific Workshops ...................................................................................... 15 Table 7. Final Definitions............................................................................................................................ 20 Table 8. Data Collection Summary ............................................................................................................. 22 Table 9. CEDARS and Other Data Referenced .......................................................................................... 24 Table 10. Format of Initial Results Provided to PAs, with Illustrative Program Results ............................ 26 Table 11. Substantively Similar Definition Properties ................................................................................ 27 Table 12. Summary of Potentially Substantively Similar or Duplicative Programs ................................... 33 Table 13. Count of Substantively Similar and Duplicative Programs By PA ............................................. 35 Table 14. Annual Potential Ratepayer Risk................................................................................................. 38 Table 15. Southern California Scheduled PASC Meetings ......................................................................... 46 Index of Figures
Figure 1. Program Screening Process Visualization ................................................................................... 27 Figure 2. Number of Mitigation Strategies Uses for Substantively Similar or Duplicative Programs ....... 36 Figure 3. Reduced Ratepayer Risk .............................................................................................................. 38 Figure 4. Duplication Spectrum 3 and 4 by Segment ................................................................................. 40 Figure 5. Duplication Spectrum 3 and 4 by Sector ..................................................................................... 42
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
5 Acronyms
Table 1. Acronyms Used in Report
BayREN Bay Area Regional
Energy Network CPUC California Public Utilities Commission ED Energy Division I-REN Inland Regional Energy
Network
JCM Joint Cooperation
Memo MCE Marin Clean Energy
OP Ordering Paragraph PAs Portfolio
Administrators
PG&E Pacific Gas and Electric
Company SCE Southern California Edison Company
SoCalGas Southern California
Gas Company SDG&E San Diego Gas & Electric Company
SoCalREN
Southern California
Regional Energy
Network
3C-REN Tri-County Regional
Energy Network
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
6 Introduction and Overview
Pursuant to Ordering Paragraph (OP) 32 of Decision (D.) 23-06-055, issued on July 3, 2023, the Portfolio Administrators (PAs) are pleased to provide the California Public Utilities Commission (CPUC or Commission) with this report. The PAs represented in this report are:1 • Bay Area Regional Energy Network (BayREN)
• Inland Regional Energy Network (I-REN)
• Marin Clean Energy (MCE)
• Pacific Gas and Electric Company (PG&E)
• San Diego Gas & Electric Company (SDG&E)
• Southern California Edison Company (SCE)
• Southern California Gas Company (SoCalGas)
• Southern California Regional Energy Network (SoCalREN) • Tri-County Regional Energy Network (3C-REN)
The PAs appreciate the Commission’s interest in ensuring that ratepayer dollars are used in a prudent manner and that ratepayer-funded energy efficiency programs are designed and operated to maximize benefits to customers, communities, and the environment. Working together to complete the objective and requirements of OP 32 and its six sub-objectives provided the PAs a useful opportunity to share and discuss perspectives on program delivery, strategy, and approaches to risk mitigation.
Throughout the multi-month process of developing responses to OP 32, these discussions expanded on the existing activities PAs undergo regionally as part of the Joint Cooperation Memo (JCM) process. These activities were and continue to be in line with the Commission’s maintained “preference for PAs to work collaboratively not only to minimize duplication in non-hard-to-reach customer populations but importantly to strive toward effective regional strategies and complementary program offerings; to be clear, PAs should communicate regularly in the course of administering their portfolios and preparing applications for future cycles.”2
In explaining its rationale for requiring the PAs to develop and submit an advice letter as detailed in OP 32, the Commission discussed the need for specific examples or scenarios that indicate the possible risk of duplicative programs. The Commission also stated that “we are not wedded to establishing formal 1 San Diego Regional Energy Network (SDREN) was authorized on August 7, 2024 via Commission D.24-08-003.
Due to the timing of the approval, SDREN’s ten programs are not included in the program overlap analysis. In addition, Rural REN’s programs are not included in the program overlap analysis due to pending Commission action as of the time of the development of the analysis.
2 D.23-06-055 at 89.
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
7 guidance if the PAs demonstrate they have a process or protocol to, among themselves, effectively mitigate or minimize ratepayer risks associated with duplicative or overlapping programs.”3 It is the PAs’ aim that the information provided in this report will demonstrate that instances of program duplication or overlap,4 which could potentially lead to ratepayer risk (ratepayer funding that does not provide value) have been identified and with ongoing collaboration should present reduced risk.
Furthermore, the amount of ratepayer dollars at risk that were, as calculated as part of using the methodology in this exercise report, is considered low and the PAs have mitigation strategies and actions in place to minimize the potential for ratepayer risk from these programs. In the limited instances where unmitigated potential duplication has been identified by a PA, this report afforded the opportunity for in-depth discussions regarding mitigation strategies. PAs will continue to share and refine mitigation strategies to further the progress made through the joint efforts undertaken in the development of this report.
Background Information
Decision Requirements
OP 32 of D.23-06-055 requires: “By no later than September 1, 2024, the portfolio administrators (PAs) shall submit a joint Tier 2 advice letter to provide information on substantively similar programs, and steps they have taken and will take to mitigate or minimize ratepayer risk of program overlap and duplication. The joint advice letter must provide:
a) A comprehensive list of any substantively similar ratepayer-funded energy efficiency programs among the PAs.
b) A clear statement of the issues or problems that result from program offerings identified in item (a). c) Definitions or clarifications of any jargon that PAs suggest specifying, in order to have a shared understanding of the issue or problems associated with substantively similar ratepayer-funded energy efficiency programs among different PAs. For example, the joint advice letter should propose definitions for “overlap,” “duplication,” and “precedence.” d) Description of the risk to ratepayers of ratepayer-funded energy efficiency programs that “overlap” or are otherwise “duplicative” (as defined in response to Item (c), above) in some substantive way.
What is the estimated dollar value of the risk? Describe how the value was calculated or assessed. e) Description of the actions, measures, etc. that PAs have taken thus far to identify and mitigate or minimize risks to ratepayers of substantively similar ratepayer-funded energy efficiency programs 3 Id. at 87.
4 Please see the sections “Identify and Define Relevant Terms and Criteria” and “Definitions” for delineation of the terms program duplication and program overlap. The terms duplication and overlap have the same meanings as program duplication and program overlap throughout this document, respectively.
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
8 among different PAs, and any other issues identified in response to Item (b). Include, for instance, any agreed-upon criteria for determining program “overlap,” “duplication,” and/or “precedence,” and what steps have been taken by each PA in cases where “overlap” or “duplication” was identified.
Explain the effectiveness of each of these measures, actions, etc. in mitigating ratepayer risks, and identify and describe what issues remain unresolved.
f) Description of how the PAs will effectively mitigate or minimize ratepayer risks associated with similar ratepayer-funded energy efficiency programs among different PAs through the joint cooperation memoranda or any other agreed-upon process or protocol.” On August 16, 2024, SoCalREN, on behalf of the PAs, requested a 30-day extension to October 1, 2024, pursuant to Rule 16.6 of the CPUC’s Rules of Practice and Procedure. On August 21, 2024, the Executive Director of Energy Division (ED) granted the requested extension.
Compliance with Decision Guidance
OP 32 contains six advice letter requirements (a) through (f), some of which in turn have multiple components. Given this multi-part directive, responses can be found in multiple sections of the report.
The below table is designed to assist the reader in locating which sections of the report the PAs’ responses to each requirement can be found.
Table 2. OP32 Assignment Compliance Table
OP 32 Advice Letter Requirement Compliance
a) A comprehensive list of any substantively similar ratepayer-funded energy efficiency programs among the PAs. • In addition to the Excel Workbook
included as the Appendix to this report, a
comprehensive list is included in
• Table 12 on page 33.
b) A clear statement of the issues or problems that result from program offerings identified in item (a).
• This statement is located in the Issues
Resulting from Analysis and Unresolved
Issues section beginning on page 43.
c)
Definitions or clarifications of any jargon that PAs suggest specifying in order to have a shared understanding of the issue or problems associated with substantively similar ratepayer-funded energy efficiency programs among
different PAs. For example, the joint advice letter should propose definitions for “overlap,” “duplication,” and
“precedence.”
• Definitions for the four terms the PAs
elected to define are included in the
Definitions section of this report,
beginning on page 19.
d)
Description of the risk to ratepayers of ratepayer-funded energy efficiency programs that “overlap” or are otherwise “duplicative” (as defined in response to Item (c), above) in some substantive way. What is the estimated dollar value of the risk? Describe how the value was calculated or assessed. • This description is located in the Issues
Resulting from Analysis and Unresolved
Issues section beginning on page 43.
• The estimated dollar value is noted in the
Appendix workbook as well as in the
Findings and Analysis section of this
report beginning on page 32.
• The process by which the value was
calculated is included in the Methodology
section, beginning on page 29.
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
9 OP 32 Advice Letter Requirement Compliance
e)
Description of the actions, measures, etc. that PAs have taken thus far to identify and mitigate or minimize risks to
ratepayers of substantively similar ratepayer-funded energy efficiency programs among different PAs, and any other
issues identified in response to Item (b). Include, for instance, any agreed-upon criteria for determining program “overlap,” “duplication,” and/or “precedence,” and what steps have been taken by each PA in cases where “overlap” or “duplication” was identified. Explain the effectiveness of each of these measures, actions, etc. in mitigating ratepayer risks, and identify and describe what issues remain
unresolved.
• Description of actions, measures to
mitigate risks is included in Mitigation
Options and Scoring on page 28.
• Criteria for agreed-upon terms are
included on page 19.
• Steps taken to address overlap and
duplication as well as effectiveness of
mitigation measures are noted on page 32
and in Appendix A.
• Issues that remain unresolved are noted
in the Issues Resulting from Analysis and
Unresolved Issues section.
f)
Description of how the PAs will effectively mitigate or minimize ratepayer risks associated with similar ratepayer-funded energy efficiency programs among different PAs through the joint cooperation memoranda or any other
agreed-upon process or protocol.
• A final discussion of how PAs will
continue to effectively mitigate or
minimize potential ratepayer risk is
described in the Next Steps section
beginning on page 46.
Background Context and Document Review
As the PAs embarked on the process to develop the data to comply with OP 32, they were cognizant of and, wherever possible, leveraged past Commission direction and existing coordination processes.
In D.18-05-041, which adopted the PAs’ 2018–2025 Energy Efficiency portfolios, the Commission acknowledged the potential overlaps between investor-owned utility (IOU) PAs and non-IOU PAs and directed PAs with overlapping service areas to submit annual JCMs that show how the PAs would avoid or minimize duplication for programs that address a common sector in overlapping service territories.5 In D.21-05-031 and D.23-06-055, the Commission issued updated requirements for JCMs, requiring the Regional Energy Networks (RENs) to provide additional information on certain programs meeting the hard-to-reach criterion.6
The JCM development process involves a series of collaborative meetings between PAs, where sector and program-level strategies are debated and developed. The JCMs help PAs determine—at a regional level— the best, most effective methods to avoid customer confusion and prevent negative outcomes from potential program overlap. A number of these strategies are common among PAs and may translate well to the statewide level, as noted during OP 32 collaboration meetings discussed later in this report. JCMs are submitted every two years after the approval of the last of each JCM’s PAs’ true-up advice letters and mid-cycle advice letters (as applicable).
5 D.18-05-041 at 122-123.
6 As modified in D.23-06-055 at 45-54.
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
10 Summary of Findings
In the context of OP 32, ratepayer risk can be thought of as the potential for expenditure of ratepayer funds with suboptimal value provided or created. Generally, the potential risks of program duplication include inefficiencies in resource allocation, confusion among customers, and increased costs due to duplication of efforts. Duplicative programs may also struggle with clear delineation of responsibilities, leading to gaps in service delivery or redundant work. The PAs developed definitions and a Duplication Spectrum to systematically quantify this risk.
The PAs leveraged the California Energy Data and Reporting System (CEDARS) and the PAs’ 2024-2025 budget filings to methodically review all programs for similarity, as defined and used in the Duplication Spectrum. Upon identifying a program that was potentially Substantively Similar or Duplicative7 to another program, PAs catalogued which mitigation strategies – from an agreed upon list - and actions were in place to mitigate against potential risk. The mitigation methodology developed and applied by the PAs worked such that if a PA affirmed that it uses one or more of the risk mitigation methods, then the calculated risk to ratepayer dollars was reduced, in some cases to zero, even in instances of program duplication.
The PAs started their analysis with the 599 PA program IDs8 found in the 2024-2025 budget filings that had a status of "New," "Active," or "Transitioning." From there, PA-established filters flagged programs meeting the criteria for the defined terms Similar and Program Overlap. 9 The PAs reviewed the results for
accuracy, and further screened the programs to determine if they met the criteria for the defined terms Substantively Similar or Duplicative.
Seven of the nine PAs self-identified at least one program within their portfolio that was Substantively Similar or Duplicative program with another PA’s program. In total, 27 programs were identified as being Substantively Similar or Duplicative with another program. The PAs calculated that if no mitigation strategies were in place, these 27 programs would contribute to $4,900,000 in annual risk to ratepayers.10 However, due to the risk mitigation strategies already in place, the calculated risk of these programs is reduced by $3,400,000 or 69%, leaving $1,500,000 in calculated annual risk that is not currently eliminated via a mitigation strategy. This unmitigated risk represents 0.15% percent of the total annual average authorized energy efficiency budget for the PAs. 7 The terms “Substantively Similar” and “Duplicative” are key terms defined below. 8 These program IDs include those assigned to portfolio or program support PA costs and separate PA-specific instances of statewide programs, therefore the number of program IDs represented here are inclusive of all program IDs in use by PAs.
9 See discussion below in the sections entitled “Identify and Define Relevant Terms and Criteria” and “Definitions”. 10 Quantified values are rounded. Non-rounded values are included in the Appendix.
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
11 Risk was distributed among various PAs, segments, and sectors. Resource Acquisition had the most Substantively Similar and Duplicative programs by segment, while the Residential and Workforce, Education & Training (WE&T) sector had the highest number by sector.
The PAs look forward to continuing to coordinate through the JCM process and other meetings to refine their approaches, share best practices, and advance mitigation strategies.
Project Approach
In 2023, the California PAs began to discuss OP 32 to determine the most efficient yet impactful approach to completing the six OP 32 sub-objectives. SoCalREN and SDG&E representatives volunteered to co-chair the process and to lead a Steering Committee established to oversee the effort and ensure the resulting output complies with Commission guidance. SoCalREN hired a consultant, ICF (Facilitator), to facilitate discussions and work with the PAs to develop necessary methodologies and analyses as required in the six OP 32 sub-objectives.
The project approach, which comprised eight stages, is summarized in Table 3, followed by additional narrative details related to each stage. As developing, vetting, and discussing Substantively Similar or Duplicative programs and mitigation strategies is not a linear process, the PAs concurrently collaborated on several stages. Additional details are provided in the Methodology section.
Table 3. Project Approach Summary
Stage Objective Approach Outputs
1. Establish
Steering
Committee
Identify process/timelines
for data collection,
analysis and reporting
• Hold weekly meeting
• Project Process
• Project Timeline
2. Identify and
Define
Relevant
Terms and
Criteria
Identify, refine and seek
consensus of terms
needing definition, and
developing criteria for
progressive levels of
program overlap risk
• Hold multiple facilitated
workshops with
representatives from all
PAs
• Identification which
terms require definition.
• Initial definitions
formulated, with criteria
components.
• Feedback on definitions
• Consensus on final
definitions
3. Development
of Program
Matrix Tool V1
Consolidate CEDARS
data to identify to what
degree Substantively
Similar or Duplicative
risk criteria could be
automatically identified
via the criteria identified
when defining terms
• Identify relevant CEDARS
fields available which align
with definition criteria
• Extract CEDARS data
• Develop advanced program
matching macro to identify
which programs might
overlap with which other
programs
• A matrix of all active
programs within the
CEDARS database
• Identification of which
programs indicated
matching criteria with
other programs at level 1
or 2 of the Duplication
Spectrum (defined later
in report)
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
12 Stage Objective Approach Outputs
4. Explore
Programs by
Sector for
Overlap
Review the programs by
sector with potential
overlap and identify any
general overlap issues that
may be common to
programs within a
specific sector
• Generate sector specific
reports of potential
overlapping programs by
sector
• Workshops for each of
eight key sectors to review
potential overlap issues
• Eight specific Excel
reports filtered to
identify at risk programs
in each sector
• PA feedback on general
and program specific
duplication risks
5. Development
of Program
Matrix Tool V2
Consolidate feedback
from workshops to
develop a more detailed
Program Matrix tool for
PAs to quantify specific
programs at risk of
overlap within each PAs
portfolio, and identifying
any mitigation approaches
employed
• Refresh export from
CEDARS data
• Incorporate risk mitigation
methodology and risk
scoring
• Update Excel tool / filter
structure
• Import and structure
filtering in Excel to identify
programs that potentially
aligned with criteria
• A matrix of all active
programs within the
CEDARS database
• Identification of which
programs indicated
matching criteria with
other programs at level 1
or 2 of the Duplication
Spectrum
• Itemized identification
of each program that
might overlap with
another
6. Explore,
Qualify, and
Quantify
Program
Overlap Risk
by PA
Review, verify and detail
programs that have been
identified as at risk of
being Substantively
Similar or Duplicative
within their program
portfolio
• Develop reports of
programs identified as
potentially at risk based on
duplication criteria 1 and 2
were generated specifically
for each PA
• Reports were distributed for
review
• Reports for each PA
filtering only for
programs that met level
1 or 2 criteria
• Identification by PAs of
any programs also
meeting the level 3 or 4
criteria, the nature of the
duplication, methods of
duplication risk
mitigation and the scale
of the budget that may
be at risk
• A worksheet containing
a calculation model
based on PA inputs that
would calculate a
ratepayer dollar risk
7. Aggregate and
Analyze Data
Aggregate completed
worksheets for quality
review and insights
analysis
• Assemble Excel reports
returned to Steering
committee
• Aggregate, filter, and
review data manually
• Reviewed Excel
worksheets where PAs
have identified if any
programs meet the level
3 and/or 4 duplication
criteria, plus
corresponding additional
details and mitigation
strategies
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
13 Stage Objective Approach Outputs
8. Produce,
Review and
Finalize
Report and
Advice Letter
Draft final report and
advice letter in
compliance with OP 32
for review and approval
• Draft and complete
multiple rounds of review
• Circulate draft and final
advice letter and report
for review and approval
As displayed in Table 3 above, the project approach consisted of eight primary stages, which iterated on and sometimes concurrently operated. Below are additional details on the eight stages, including key dates, outcomes of workshops and meetings, and other relevant information.
Establish Steering Committee
The purpose of this stage was to identify and delegate to a small project team to move the process through the various stages of discussion, data collection, workshops, milestones, and final output. As noted above, SoCalREN and SDG&E volunteered to lead the Steering Committee and oversee a third-party facilitator (ICF) who would conduct meetings and coordinate assignments. All PAs were actively involved in developing the third-party facilitator scope of work. The Steering Committee met on a bi-weekly basis to track progress toward OP 32 sub-objectives and overall task completion and to address any issues faced by PAs in completing the OP 32 sub-objectives.
Identify and Define Relevant Terms and Criteria
This stage's purpose was to identify, define, discuss, revise, and achieve consensus on the definitions and criteria for progressive levels of program overlap risks in consultation with representatives from all relevant PA teams. Rather than each PA working independently to develop terms and then debate them in a single meeting setting, the PAs took a structured, facilitated approach that allowed multiple iterations of definitions and the ability for PAs to come to an agreement on key “bedrock” terms as described in the Definitions section on page 19. Key activities conducted as part of completing this process step and associated dates are detailed below in Table 4.
Table 4. Activities to Identify and Define Relevant Terms No. Activity Date
1 Identify initial potential terms and criteria samples drawn from representative documents for the purpose of generating feedback and discussion.
April/May
2024
2 Create and email definitions sample worksheets to identified PA teams for review. May 2024 3 Conduct first discussion workshop with PA teams. 5/16/24 4 Consolidate feedback from the workshop and transmit updated worksheets for review prior to a second workshop. 5/21/24
5 Conduct second discussion workshop with PA teams. 5/23/24 D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
14 No. Activity Date
6 Consolidate workshop input and revise definitions for distribution to PAs for third round of review. 5/30/24
7 Worksheets with comments returned by PAs. 6/10/24
8 Final definitions transmitted for review. Instructions indicated that, if there were no substantive feedback, terms would be accepted as final. 6/12/24 9 Terms accepted as Final by Steering Committee and distributed to PA representatives. 6/17/24
Development of Program Matrix Tool V1
This stage's purpose was to develop an initial data exploration tool using the finalized program overlap definitions in combination with program data from CEDARS to make it easier to identify programs at risk of significant overlap. Using the available data, this tool was limited to qualifying and filtering programs that met the criteria for Similar and/or Program Overlap. This tool was used to filter down to at-risk programs for the general sector-level workshops in the next stage. The outputs for this stage included a) extraction of all program data from CEDARS and b) the development of Program Matrix Tool V1.
Explore Programs by Sector for Overlap
The purpose of this stage was to review programs in specific sectors with PA program teams with deep familiarity with those sectors. The PA program teams worked to identify if there may be outstanding risks resulting from sector characteristics that could be addressed before or during the development of the program overlap analysis. Key activities conducted as part of completing this process and associated dates are detailed below in Table 5.
Table 5. Activities to Explore Programs by Sector for Overlap No. Activity Date
1 Identify sectors for review, referencing work completed by PAs as part of the 2024 JCM development process. June 2024
2 Using the Program Matrix Tool V1 generate reports for each sector that identified programs that qualified as either Similar or Program Overlap.
June 2024
3 Transmit both the Matrix Tool V1 as well as sector-specific reports to PAs identified for each sector. 7/17/24
4 Conduct sector specific workshops to discuss if there are any issues that may be specifically relevant to that sector. 7/22/24–7/25/24
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
15 5 Consolidate findings for analysis process. 7/26/24–8/5/24 As noted above, PAs participated in sector-specific workshops to discuss issues and topics relevant to each sector. Below, Table 6 provides a detailed overview of each workshop, including when it was held, the number of participants, and key topics and themes discussed within each workshop.
Table 6. Overview of Sector-Specific Workshops
Sector Date Persons
Attended Key Discussion Themes
Agriculture 7/22/24 25
• Duplication and Similarity
• Data Accuracy Concerns
• Methodology for Identifying Similar Programs
• Statewide Program Identification
• Concerns About Program Overlap
• Mitigation Measures for Program Duplication
• Data Sharing and Prior Participation Checks
Commercial 7/22/24 25
• Program Duplication Risk
• Statewide Programs Discussion
• Program Identification and Mitigation
• Concerns on JCM Coverage
• Action Items for Program Duplication
Cross Cutting:
Codes and
Standards
7/23/24 27
• Statewide Programs
• Program Inclusion
• JCM Coordination
• Definition of Statewide Duplication Risk
• Coordination Efforts
• IOU Coordination
Cross-cutting:
Workforce,
Education and
Training
7/23/24 31
• Program Duplication Concerns
• Data Accuracy and Program Listings
• Statewide Program Coordination
• Complementary vs. Duplicative Programs
• Coordination Among Utilities
• Concerns About Missing Programs
Public 7/24/24 31 • Meeting Objective
• Program Review Process
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
16 Sector Date Persons
Attended Key Discussion Themes
• Coordination and Mitigation
• Program Solicitation and Duplication
• Risk of Duplication
• Vendor Coordination
Residential 7/24/24 23
• Statewide Program Coordination: Data and
Duplication Concerns
• Program Listing and Data Accuracy
Statewide
Programs 7/25/24 17
• Program Overlap and Risk Management
• Statewide Program Leads and Responsibilities
• Mechanisms to Avoid Duplication
• Concerns on Program Overlap
• Statewide vs. Local Program Dynamics
• JCM Process and Statewide Programs
Cross Cutting:
Finance 7/25/24 13
• Letter Preparation Process
• Program Duplication
• Program Activation Status
• Sector-Specific Risks
• Data Accuracy and Program Status
• Concerns and Clarifications
• Process Improvement Suggestions
Workshop Outcomes
The workshops were an important starting point for developing a program overlap identification process, and for hosting productive conversations on addressing overlap through coordination and mitigation.
However, improvements to the process were necessary to provide a more cohesive methodology and produce more granular analysis. These improvements are discussed in the Methodology section. Upon concluding the workshops, the PAs came to the following conclusions and identified the following concerns.
• No Identifiable Duplication based on Sector: Using the Program Overlap Matrix V1, no sector workshop group was able to identify programs at risk of Substantively Similar or Program Duplication characteristics across a specific sector. Further refinements to the Matrix in V2 helped clarify and identify potential overlap, assisting PAs later in identifying potential overlap. • Easier for PAs to Review Programs Sorted by PA Portfolio: Following the conclusion of the sector-specific workshops, the PAs determined that it would be more productive to review D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
17 programs sorted by PA and filtered for level 1 and 2 program overlap risk on the Duplication Spectrum. These changes were incorporated in the Program Overlap Matrix V2. • Need for a Comparison Document Across All Programs: PAs agreed across several workshops to develop a second version of the Program Matrix Tool to support their own review of their portfolios for programs that were flagged as potentially meeting the definitions of Substantively Similar or Program Duplication.
• Utilizing Existing Data: The PAs considered if it was possible to utilize data sources beyond CEDARS to eliminate duplication and identify programs needing clarification. For the moment CEDARS remains the best source of data for this analysis. • Concerns with Data Accuracy: The PAs contended with data accuracy issues, including missing territories and program details, which could prevent identification of similar programs. • Need for Methodology for Identifying Similar Programs: The need for a structured approach to categorize programs based on risk levels of duplication and similarity. The PAs agreed to improvements to the identification process, which are discussed in the Methodology section. • Challenges with Statewide Program Identification: There are challenges in consolidating and accurately identifying statewide programs to avoid redundancy. There is a need for better consolidation and accurate identification of statewide programs and their interaction with non-statewide programs. • Mitigation Measures for Program Duplication: The PAs acknowledged they must continue to communicate and improve upon risk mitigation strategies if programs are potentially Substantively Similar or Duplicative and the risk cannot be mitigated, and conducting prior participation checks to mitigate duplication risks.
• Importance of Coordination Efforts: PAs agreed to the importance of coordination, formal agreements, and data sharing to prevent program duplication was a frequent topic. • JCM Coverage and Coordination Importance: The PAs discussed gaps in JCM coverage, particularly for statewide programs. The PAs identified the need for comprehensive JCM coverage to mitigate duplication risks. Solutions were discussed including the potential for future JCMs to include all statewide programs, regardless of whether the PA administering the statewide program shares overlapping territory with another PA.
• Program Inclusion and Activation Status Challenges: Ensuring comprehensive program coverage and understanding the activation status of programs was challenging in some cases and may limit accurate duplication risk assessment.
• Sector-Specific Risks and Dynamics: PAs discussed the need for better use of JCMs and other discussions to understand sector-specific risks of duplication and the distinct differences between regions to address all needs effectively.
Development of Program Matrix Tool V2
The goal of this stage was to gather feedback from the sector-specific workshops, along with additional discussion points and identified issues, to create a more refined Program Matrix for each PA to evaluate their program portfolios. The process for this stage was as follows:
D.23-06-055 Ordering Paragraph 32 Report
October 1, 2024
18 • Review feedback from sector meetings and conversations to date to identify what additional data points and clarifications are necessary to enable collection of needed data. These issues were highlighted above in the Workshop Outcomes section.
• Develop criteria to streamline the process for PAs to review potentially overlapping programs. • Import CEDARS data, validate and correct for errors and omissions identified by PAs in the previous version of the tool.
• Integrate JCM data where available.
• Develop and configure the Program Matrix Tool V2 to only display programs that qualified as either Similar or Program Overlap.
• Each PA utilize the Program Matrix Tool V2 to validate Similar or Program Overlap programs within their portfolios.
Explore, Qualify, and Quantify Program Overlap Risk by PA In order to streamline each PA’s program overlap identification process, the steering committee determined that individual reports should be generated for each PA that are filtered for only those programs that are relevant to each PA. The process for this stage was as follows:
• Program Matrix Tool V2 was pre-filtered for each PA so that only programs that qualified as either Similar or Program Overlap were displayed based on available data. • Pre-filtered Tools were only emailed to PAs if the filtering process generated results showing potentially Similar or Program Overlap programs.
• PAs reviewed programs and identified if any programs demonstrate further overlapping characteristics and qualify as Substantively Similar or Program Duplication.
If a program met either or both sets of criteria for Substantively Similar or Program Duplication, the PAs were instructed to input a qualitative discussion of the nature of the duplication, select the appropriate methods of mitigation in place (if relevant), and score the effectiveness of the selected mitigation methods. The Tool then calculated the dollar risk resulting from identified Substantively Similar or Program Duplication programs based on the identified methods of overlap mitigation for each program.
Additional discussion of this methodology is found in the Methodology section of this document.
Aggregate and Analyze Data
The purpose of this stage was for the Facilitator to aggregate and review programs that PAs identified as qualifying as Substantively Similar or Program Duplication, the identified methods of overlap mitigation, and the estimated dollar risk of overlap to identify key insights. The process for this stage was as follows:
• Facilitator collected all PA reports.
• Facilitator review comments and issues identified in returning emails from PAs. • Facilitator consolidated programs identified as qualifying as Substantively Similar or Program Duplication into single data set.
• Steering Committee and Facilitator analyze results to determine data insights.