Topic:"Evidence Requirements" in M12249

3.1 2026 DSM EXTENSION ASSUMPTIONS: INCIDENTAL IMPACTS To estimate the incidental low-income and equity impacts for the 2026 DSM Extension from non-targeted programs, E1's historical low-income and equity reporting relies on information ab...

AI summary The document discusses estimating incidental low-income and equity impacts for the 2026 DSM Extension using E1's historical data from 2023-2024, as actual project details are not available through modeling. Table 2 outlines assumptions and calculations for non-targeted program components.

4.4 MODEL CHANGE LOG - During the 2022 Historical Rate and Bill Impact Analysis proceeding, Synapse requested that E1 document - any formula changes in the RBIA model and in its Decision Letter dated February 24, 2023, the Board - directed...

AI summary During the 2022 Historical Rate and Bill Impact Analysis proceeding, Synapse requested E1 to document formula changes in the RBIA model. The Board directed E1 to create a change log for the E1 RBIA model and NS Power rate model to track updates, with a change log tab added to both models.

"E1 Data Inputs" tab This tab includes information provided to NS Power by E1 on DSM Program measures and avoided unit costs, all of which are used in determination of class unit costs and revenues.

AI summary The 'E1 Data Inputs' tab details data provided by E1 to NS Power regarding DSM program measures and avoided unit costs, which are essential for calculating class unit costs and revenues.

Vision Statement To be a world leader in developing truthful measurement and useful results; to support the development of efficient, ethical, and effective practices, sustained economically; to advance human development. To improve the qu...

AI summary The Vision Statement aims to lead globally in truthful measurement and impactful results, promoting efficient, ethical, and economically sustainable practices. It emphasizes advancing human development and improving quality of life amid rapid climate change.

the others. 52 Table 86, Evaluated 2024 Residential Behavior Net Electrical Energy Savings, Econoler Residential Behavior Evaluation, Existing Programs, P. 195. program, since there are no physical measures installed, we have only the mete...

AI summary The evaluation of residential behavior programs' energy savings highlights concerns about the validity of aggregated GWh results. With large sample sizes, statistical significance is less meaningful, and effect size is emphasized. The 6.27 GWh annual savings are questioned for practical impact on power plant decisions due to their distribution across 80% of residential customers without physical measures.

s to Improve the Conduct and Interpretation of Quantitative Science, March 7, 2016 [(www.amstat.org/asa/files/pdfs/p-valuestatement.pdf)](https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf). At the utility system level, the effect...

AI summary The analysis critiques the statistical significance of a 6.27 GWh effect size for a program, arguing it is negligible compared to NSP's 2024 system requirement of 11,326.3 GWh (0.06%). Large sample sizes are noted to distort statistical significance, rendering savings claims at both household and system levels invalid without further evidence from NSP.

herently inaccurate". According to the Universal Methods Project (UMP) protocol[60,](#page-69-3) they function based on correlation with the sound produced by a leak rather than by direct measurement. Ultrasonic leak detectors are an effec...

AI summary Ultrasonic leak detectors are criticized for being inherently inaccurate due to reliance on sound correlation rather than direct measurement. The Universal Methods Project (UMP) protocol advocates for standardized leak-down tests pre- and post-repair. Savings from sites without independent evaluations are recommended to be rejected due to methodological flaws.

Review of Causality of the Issue for Industrial Customers

AI summary The Nova Scotia Utility and Review Board (NSUARB) is conducting a review to assess the causality of the issue for industrial customers. The analysis focuses on determining the relationship between specific factors and the impact on industrial electricity consumers, though detailed arguments or findings are not provided in the text.

024 DSM Measure Assessment[.9](#page-7-4) These calculations were established based on a comprehensive review of other Technical Reference Manuals by the Evaluator to validate the approach used by E1. The evaluation approach for compressed...

AI summary EfficiencyOne (E1) explains its 2024 compressed air leak audit evaluation methodology, citing validation from Wisconsin, Minnesota, and other state Technical Reference Manuals (TRMs). The approach includes savings verification, project file reviews, site visits, and interviews for major projects, with specific references to E1's 2024 DSM Programs Evaluation Reports (M12186, Exhibit 2).

Request IR-3: Refer to M12249, Exhibit E-2, 2024 Peach Report, Section IX, General Recommendations, including Recommendation SVR24-G-4, which states at page 18: SVR24-G-4. The Evaluator should include full worksheets or computer printout i...

AI summary Peach requested detailed significance test information from the evaluator, but the consultant provided only partial responses. Peach noted that power values of 100% indicated statistical significance was not reliable. The recommendation SVR24-G-4 requires full test details for future evaluations to ensure transparency. The evaluator's incomplete responses were deemed sufficient for analysis.

f the program administrator. The verification consultant reports directly to - the NSEB for additional oversight on measurement, evaluation, and performance issues that the - evaluator might overlook. - The roles of the independent evaluat...

AI summary The text outlines roles for program administrators, independent evaluators, and verification consultants in enhancing program effectiveness through feedback, evidence-based decisions, and SWOT analysis. Oversight by the Nova Scotia Energy Board (NSEB) ensures measurement and performance accountability, with a focus on improving efficiency and strategic planning.

Request IR-12-a : Please provide the source for the referenced customer data supporting the Verifier's findings.

AI summary The document requests the source of customer data supporting the Verifier's findings in Request IR-12-a, emphasizing the need for transparency and verification of data used in regulatory proceedings. This request aims to ensure the reliability and legitimacy of the Verifier's conclusions by requiring the disclosure of the underlying data sources.

Response IR-12-a: - The source for the referenced customer data supporting the Verifier's findings is the evaluation - consultant's 2024 final report and E1 tracking sheet ex-ante claim for the two locations.

AI summary The response identifies the evaluation consultant's 2024 final report and E1 tracking sheet as the sources for customer data supporting the Verifier's findings.

Request IR12-b: - Please provide calculations along with relevant data used to establish the leak rate at these - locations, and to support the statement that savings at these locations were a substantial portion - of total energy demand f...

AI summary The request asks for calculations and data to determine leak rates at specific locations and to substantiate the claim that energy savings from these locations were a significant portion of total compressed air demand.

5 Request IR-12-e: 4 8 12 - 6 In reference to the above statement what constitutes "substantial", and upon what - 7 methodological basis does the Verifier base its understanding of this term?

AI summary The text is part of a regulatory proceeding under Request IR-12-e, seeking clarification on the definition of 'substantial' and the methodological basis used by the Verifier to interpret this term.

6 Q. DO YOU INTERPRET THE EVALUATION RESULTS IN THE SAME WAY AS 7 MR. PEACH? 8 A. No. While I agree that there are observable results, I disagree that they are so small as to 9 warrant no practical value. Mr. Peach's description of the eff...

AI summary The witness disagrees with Mr. Peach's interpretation of evaluation results, arguing that small absolute values still hold practical significance due to their impact on program participation. Mr. Peach's analysis relies on Table 2, but the witness contends his approach overlooks relative impacts.

E1 Response - E1 acknowledges the current TRC test as described, and notes Synapse's observation. Evaluating all costs - but only a subset of benefits leads to an imbalanced test that does not reflect participant benefits. E1 seeks - to ad...

AI summary E1 acknowledges the current TRC test but argues it is imbalanced by considering only a subset of benefits. E1 proposes a new BCA test in its pending Application (M12282) to assess DSM Plans by incorporating both utility and non-utility system impacts.

E1 Response E1 accepts Synapse's observation that locational DSM can be very effective to assist with grid constrained areas. E1's energy efficiency and demand response program delivery can be targeted specifically to customers, however, t...

AI summary E1 agrees with Synapse's view that locational demand-side management (DSM) can effectively address grid constraints but emphasizes that targeting programs requires detailed customer data from NS Power. While acknowledging NS Power's provision of constrained substation lists, E1 requests customer mapping to those substations for improved program efficacy.

2. Sample Size and Statistical Significance The Peach Report states as follows in relation to the use of statistical significance testing for large sample sizes:[6](#page-23-0) Use of statistical significance testing for very large sample...

AI summary The Peach Report critiques the use of statistical significance testing with large sample sizes, noting that these tests were developed for smaller samples. It highlights a tension between the benefit of larger samples in enhancing statistically significant results and the questionable applicability of significance testing with very large samples.

Econoler Response: Econoler respectfully disagrees with the statement that a statistical test will almost always demonstrate a significant difference with large sample. While it is true that a large sample size increases ability to detect...

AI summary Econoler disputes the claim that large sample sizes always detect significant savings, arguing that they reduce standard errors and increase statistical power but cannot detect non-existent effects. They reference studies and exhibits (e.g., Peach Report, NREL protocols) to support their position on statistical methodology in energy efficiency evaluations.

ntrol\_Pilot\_Evaluation\_Report\_Final\_(003).pdf](https://www.calmac.org/publications/PY2022-2023_Smart_Thermostat_Control_Pilot_Evaluation_Report_Final_(003).pdf) (last accessed 04-10-2024). Navigant, prepared for Eversource MA, Eversou...

AI summary The text references multiple evaluation reports on residential demand response programs, including smart thermostat pilots by Navigant, EPRI, and CADMUS. These studies assess direct load control and demand response initiatives, emphasizing aggregated data over device-level metrics to account for heating system interactions in homes.

Conclusion Econoler disagrees that the 2024 evaluated available DR capacity for Residential and BNI DR should not be accepted and disagrees with the recommendation that these program components should have been flagged for not producing pr...

AI summary Econoler disputes the recommendation to flag DR programs, arguing statistical significance, not practical effect size, should validate impact evaluations. The 2024 evaluation used valid sample sizes and followed industry best practices. The Peach Report recommends analyzing DR program importance, roles of NSP and Efficiency Nova Scotia, and clarifying program benefits for utility operations.

Econoler Response: As detailed in Item 5 above, Econoler considers that there was no restriction or limitation on the scope of the project reviews required for the impact evaluation of the two projects being discussed herein.

AI summary Econoler asserts that the project reviews for the impact evaluation of two projects were not restricted in scope, as outlined in Item 5. The response emphasizes the absence of limitations on the scope of analysis required for the evaluation.

NON-CONFIDENTIAL INFORMATION REQUESTS To: EfficiencyOne James Gogan, Counsel McInnes Cooper [[email protected]](mailto:[email protected]) From: Nova Scotia Energy Board Board Staff Responses Due : Wednesday, June 25...

AI summary The Nova Scotia Energy Board has issued a non-confidential information request to EfficiencyOne, with responses due by June 25, 2025. Contact details for Steve Pronko (Director, Electrical Advisory Services) and Crystal Henwood (Clerk of the Board) are provided for inquiries.

Request IR-11: Please provide the Statistics Canada data and related calculations referenced in the footnote on page 5 of 25 of E1's Evidence. Document: 321929 Date Filed: June 4/25 NSEB (E1) Page 4 of 6

AI summary Request IR-11 seeks Statistics Canada data and calculations referenced in a footnote on page 5 of E1's Evidence. The document, filed by NSEB (E1) on June 4/25, is part of a regulatory proceeding, requesting specific data for review.

Request IR-3: Refer to M12249, Exhibit E-2, 2024 Peach Report, Section IX, General Recommendations, including Recommendation SVR24-G-4, which states at page 18: SVR24-G-4. The Evaluator should include full worksheets or computer printout i...

AI summary Request IR-3 references M12249, Exhibit E-2, and the 2024 Peach Report, specifically Recommendation SVR24-G-4, which mandates inclusion of detailed significance test worksheets. It asks whether Peach confirmed missing information from the Evaluator and EfficiencyOne, if Peach requested it, and requests an example of required worksheet formatting from another reviewed report.

Request IR-01: Request IR-02: - Please provide a copy of the scope of work for which Gil Peach & Associates LLC ("Verifier") was - retained in relation to its preparation of the 2024 Verification Report. Please confirm whether the Verifier...

AI summary The request seeks the scope of work for Gil Peach & Associates LLC as the Verifier for the 2024 Verification Report and whether their scope included planning or policy-related tasks.

Request IR-12: - 2 Reference: page 64-65 of the 2024 Verification Report (Section X.-K., Evaluation Issue 3-a), the - 3 following was stated: "Observations of total claimed savings for the two locations showed that the cumulative annual cl...

AI summary The Verifier's findings indicate that two facilities' compressed air systems had unusually high leakage rates, with savings claims being a substantial portion of total energy demand. The request seeks data sources, calculations, comparisons to similar facilities, and methodological explanations for these conclusions.

NON-CONFIDENTIAL (a) Please provide leakage rates and claimed savings for the relevant years to support this statement.

AI summary The text requests EfficiencyOne to provide leakage rates and claimed savings data for relevant years to support a statement.

Submissions The Consumer Advocate has reviewed the evidence filed in this matter, including E1's application and reply evidence, as well as the evidence filed by Synapse and GEEG. On review, the Consumer Advocate sees no basis to oppose th...

AI summary The Consumer Advocate reviewed evidence in the proceeding, including E1's application and responses from Synapse and GEEG, and found no basis to oppose the requested extension. The Advocate confirmed E1's acceptance of certain GEEG recommendations and expressed satisfaction with the responses.

5.3 Savings and Verification Report Recommended Disallowances [50] Dr. Gil Peach, Board Counsel's consultant, recommended that savings from the residential behavioural program, the residential and BNI demand response programs, and the comp...

AI summary Dr. Gil Peach recommends disallowing savings from specific DSM programs due to insufficient independent evaluation, while Econoler defends its methodology as accurate and practical. Disputes focus on compressed air leak audit projects and adherence to the UMP Protocol.