Topic:"Program Evaluation" in M12932

Summary of Year Four Evaluation Results The Year 4 Report is the first EM&V conducted internally by NS Power. Overall, the findings indicate that the TVP Pilot continues to achieve peak load reductions in response to time‑varying price sig...

AI summary The Year 4 Evaluation of the Time-Varying Pricing (TVP) Pilot shows continued peak load reductions, especially in residential and commercial sectors. Participation has grown, and the report highlights statistically significant load shifting for commercial tariffs. NS Power faced delays due to a cyber incident and has requested further extensions for filing the report. The evaluation was conducted internally with advisory support from Econoler and includes stakeholder input and appendices with detailed findings.

Summary of Deliverable / Commitment Detail & Context Source 1 Initial Due by / Completed on Status Refer to File YR4 Report Report on the Year Four findings no later than 31 Jul 2025 File the annual Evaluation, Measurement and Verification...

AI summary The document outlines deliverables for the Year 4 report, including the submission of findings and an EM&V report by July 31, 2025. It also includes commitments by NS Power to analyze system margin forecasts and CPP events and to report on stakeholder engagement related to TVP options for small businesses.

Summary of Deliverable / Commitment Detail & Context Source 1 Initial Due by / Completed on Status Refer to Continue to file annual Evaluation Reports Maintain the established TVP Report and EM&V processes File an annual EM&V repor...

AI summary The document outlines commitments related to the continuation of annual Evaluation Reports and the planning of a MURB Industry Group Session. These efforts aim to support the adoption of TVP rates by MURB customers through incentives and rebates, with a focus on maintaining established processes and engaging stakeholders.

Introduction Time-Varying Pricing (TVP) is a scalable customer-focused demand response (DR) program. In Nova Scotia, the TVP pilot has grown significantly since it began in November 2021. Since its inception, the TVP pilot has offered two...

AI summary Time-Varying Pricing (TVP) is a demand response program in Nova Scotia that offers customers rate choices to shift electricity usage from peak periods. The TVP pilot, launched in 2021, includes multiple tariffs and rate classes. Phase 4 introduced a new multi-unit residential building (MURB) TOU Pilot Tariff, which is being evaluated as part of the TVP Phase 4 EM&V report.

Evaluation Scope This evaluation pilot period (i.e. Phase 4) analyzes advanced metering infrastructure (AMI) data from November 1, 2020 – March 31, 2025. Each Phase, the pilot period shifts by 1-year; however, the prepilot period remains s...

AI summary The evaluation pilot period (Phase 4) analyzes AMI data from November 1, 2020, to March 31, 2025. The pilot period shifts by one year for each phase, but the prepilot period remains static. Evaluation metrics are categorized into 'Load & Usage' and 'Economic', with details provided in Table 4.

The methodology is primarily composed of two components: control group selection and regression modelling. Control group selection includes matching treatment customers with control customers based on a similar location (i.e. neighbourhood...

AI summary The methodology for evaluating program effectiveness involves selecting a control group based on location, heating type, and participation in other programs, and using regression models like DiD and DDD to analyze load profiles and TVP price signals. Commercial TOU and CPP tariffs use a semi-DiD approach due to challenges in control group selection.

Energy Savings Programs E1 program participation[15](#page-127-1) and evaluation reports[16](#page-127-2) are used for control group selection balancing. Refer to [Attachment III: Methodology](#page-126-0) (section III.2.4 E1 [Program Bala...

AI summary The document discusses the E1 program's participation and evaluation reports used for control group selection balancing. It highlights the Eco Shift program, which uses behind-the-meter technologies like smart thermostats and EV charging to manage load during events. The program's impact is analyzed in sections related to time-varying pricing (TVP) and critical peak pricing (CPP).

III.2 Control Group Selection Control group selection in Phase 4 is comprised of four modified elements that will improve the evaluation relative to previous iterations. Changes are summarized in [Table 66](#page-128-1).

AI summary Phase 4 of the control group selection process includes four modified elements designed to improve the evaluation compared to previous iterations, as summarized in Table 66.

III.3.2 Commercial TOU & CPP This section presents the methodology used to determine the load, usage, and economic impact of the commercial TOU and CPP tariffs. The estimates for load and economic impacts are rounded and reported according...

AI summary This section discusses the methodology for evaluating the load and economic impact of commercial Time-of-Use (TOU) and Critical Peak Pricing (CPP) tariffs. It uses a Mixed-Effects regression analysis and a semi-DiD approach due to the absence of a control group and small sample sizes, which affect the precision and reliability of the estimates.

Attachment IV: Historical Methodological Changes All methodological modifications were made through collaboration with stakeholders and/or consultants. These changes and their progression have been summarized with two purposes. - 1. Inform...

AI summary Attachment IV discusses historical methodological changes made through stakeholder and consultant collaboration. The changes aim to inform readers about biases in comparing evaluation reports and document progress in rate innovation and evaluation methodology from the TVP pilot program.

IV.1.2 E1 Program Balancing (Phase 1) No balancing for E1 program effects was completed. As a result, the effects associated with E1 programs were not controlled for and could influence TVP EM&V results. Subsequent evaluations (i.e. Phase...

AI summary No balancing for E1 program effects was completed in Phase 1, which may have influenced TVP EM&V results. Subsequent phases have addressed this by balancing control group selection for E1 participation.

Agenda - 1. Welcome & Introduction - 2. TVP Tariff Development Cycle - 3. Review of Year Three Evaluation Report Results Econoler - 4. Review of Draft List of Proposed MURB TOU Tariff Evaluation Metrics - 5. Draft 2024/25 Areas of Focus Wo...

AI summary The agenda outlines key discussion points for a regulatory proceeding, including the development of a Time-Varying Pricing (TVP) tariff, review of evaluation reports, proposed tariff metrics, and planning for future work areas.

MURB TOU Tariff Evaluation Metrics Approach - Evaluation, measurement and verification (EM&V) plans are roadmaps for maximizing the learnings from pilot or program deployments - o EM&V plans aim to clearly articulate the goals of the progr...

AI summary NS Power plans to file a single EM&V report by July 31, 2025, including the existing TVP Tariffs and the new MURB TOU Pilot Tariff. The report will use metrics from prior evaluations and include new metrics like revenue impact and demand charge impact to meet Board directives and Consensus Agreement commitments.

Other Board Directives for 2024/25 As part of the Board's Decision in M11822 (2024/25 TVP Application), NS Power will also: • Report on the Year Four findings and file the annual EM&V Report no later than 31 Jul 2025 As part of the Board's...

AI summary NS Power is required to report on Year Four findings and file an annual EM&V Report by July 31, 2025, as per the Board's Decision in M11822. Additionally, NS Power must analyze system margin forecasts and CPP events, review marketing efforts, assess performance of General Service Class customers, and potentially revise incentive structures for commercial TVP tariffs based on stakeholder feedback.

Comments Received December 2024 Responses Circulated January 30, 2025 Stakeholder Comment NS Power Response Consumer Advocate (CA) Results Reported for Year Three • suggests that in future evaluation reports NSP restate the initial objecti...

AI summary The Consumer Advocate suggests that NS Power should restate initial objectives in future evaluation reports and provide summary comments on whether the TVP program is meeting its goals. NS Power agrees and will include this information in the Year Four Evaluation Report. The CA also requests that participant numbers be included in results summaries, which NS Power notes are available in the TVP Year Three Report (M11823).

EM&V Income-Related Findings from Year 3 Report

AI summary The EM&V Income-Related Findings from Year 3 Report includes a figure, likely illustrating key findings related to income and evaluation, monitoring, and verification processes.

TVP Year Four Report Appendix E Attachment 3 Page 55 of 60 APPENDIX MARKETING TACTICS & RESULTS

AI summary This section of the TVP Year Four Report provides an appendix detailing marketing tactics and their results, likely outlining strategies used and their effectiveness in achieving program goals.

Stakeholder Comment NS Power Response

AI summary The table outlines stakeholder comments and NS Power's responses, though specific content is not visible due to the image placeholder. The discussion likely involves stakeholder engagement, regulatory process, and program evaluation.

Introduction Time-Varying Pricing (TVP) is a scalable customer-focused demand response (DR) program. In Nova Scotia, the TVP pilot has grown significantly since it began in November 2021. Since its inception, the TVP pilot has offered two...

AI summary Time-Varying Pricing (TVP) is a demand response program in Nova Scotia that has expanded since 2021, offering TOU and CPP tariffs across three rate classes. The program aims to shift customer demand to reduce fuel and power costs, as well as long-term infrastructure costs. Phase 4 introduces a new MURB TOU Tariff, evaluated in the TVP Phase 4 EM&V report.

The methodology is primarily composed of two components: control group selection and regression modelling. Control group selection includes matching treatment customers with control customers based on a similar location (i.e. neighbourhood...

AI summary The methodology for evaluating energy efficiency programs includes control group selection based on location, heating type, and participation in specific programs, and the use of regression models such as difference-in-difference and triple difference-in-differences to assess program impacts, accounting for variables like TVP price signals and temperature.

III.2 Control Group Selection Control group selection in Phase 4 is comprised of four modified elements that will improve the evaluation relative to previous iterations. Changes are summarized i[n Table 66.](#page-113-1)

AI summary Control group selection in Phase 4 includes four modified elements aimed at improving evaluation compared to previous iterations, as summarized in Table 66.

Attachment IV: Historical Methodological Changes All methodological modifications were made through collaboration with stakeholders and/or consultants. These changes and their progression have been summarized with two purposes. - 1. Inform...

AI summary This attachment outlines historical methodological changes made through stakeholder and consultant collaboration. The changes aim to inform readers about potential biases in comparing evaluation reports and highlight progress in rate innovation and evaluation methodology from the TVP pilot program.

IV.1 Control Group Selection Control group selection methodology is documented by the Phase, for example (Phases 1,2,3) indicates a methodology that was used in Phases 1, 2 and 3 that is no longer applicable to Phase 4 onwards unless other...

AI summary The document discusses the methodology for selecting control groups, noting that methods used in earlier phases (1, 2, and 3) may not apply to Phase 4 onwards unless otherwise specified.

IV.1.1 CPP (Phase 1) Consistent with the approved evaluation plan, the CPP evaluations used a within-subject control group methodology. In practice this meant that there was no control group for CPP customers, rather the only control was t...

AI summary The CPP (Phase 1) evaluation used a within-subject control group methodology, with treatment customers during the reference day as the only control. After the first year, Econoler recommended modifying the approach to create a control group for CPP customers, aligning it with TOU evaluations.

IV.1.2 E1 Program Balancing (Phase 1) No balancing for E1 program effects was completed. As a result, the effects associated with E1 programs were not controlled for and could influence TVP EM&V results. Subsequent evaluations (i.e. Phase...

AI summary No balancing for E1 program effects was completed in Phase 1, which may have influenced TVP EM&V results. Balancing was implemented in subsequent phases starting from Phase 2.

Introduction Time-Varying Pricing (TVP) is a scalable customer-focused demand response (DR) program. In Nova Scotia, the TVP pilot has grown significantly since it began in November 2021. Since its inception, the TVP pilot has offered two...

AI summary Time-Varying Pricing (TVP) is a demand response program in Nova Scotia that offers customers TOU and CPP tariffs. The TVP pilot has expanded since 2021, with a new MURB TOU Tariff launched in 2024. The program aims to reduce fuel and power costs by shifting demand from peak periods and is evaluated through an EM&V report.

Introduction The first year a cohort enters the TVP pilot (e.g. Cohort 2 in Phase 2), the evaluation for this cohort was limited to the Winter period (i.e. November to March). Subsequent evaluations for all cohorts include a full year (i.e...

AI summary The evaluation of the TVP pilot program for the first year of a cohort is limited to the winter period (November to March), while subsequent evaluations cover a full year (April to March). Evaluation metrics are categorized into 'Load & Usage' and 'Economic'.

III.2 Control Group Selection Control group selection in Phase 4 is comprised of four modified elements that will improve the evaluation relative to previous iterations. Changes are summarized in [Table 66](#page-113-1).

AI summary Phase 4 of the control group selection includes four modified elements aimed at improving the evaluation process compared to previous iterations, as summarized in Table 66.

The intent of using neighbourhood criteria is primarily to choose control customers who have similar housing types and environmental weather patterns relative to treatment customers. In previous evaluations, the volume of AMI data that cou...

AI summary The document discusses the use of neighbourhood criteria to select control customers with similar housing types and weather patterns to treatment customers. Previously, limited local machine computations restricted the volume of AMI data processed, but cloud scale analytics now enable a broader control selection pool. This improvement enhances match precision and allows for heating classification and Eco Shift in control group selection.

IV.1 Control Group Selection Control group selection methodology is documented by the Phase, for example (Phases 1,2,3) indicates a methodology that was used in Phases 1, 2 and 3 that is no longer applicable to Phase 4 onwards unless other...

AI summary The document outlines the methodology for control group selection, noting that methods used in Phases 1, 2, and 3 may not apply to Phase 4 onwards unless otherwise specified.

IV.1.2 E1 Program Balancing (Phase 1) No balancing for E1 program effects was completed. As a result, the effects associated with E1 programs were not controlled for and could influence TVP EM&V results. Subsequent evaluations (i.e. Phase...

AI summary No balancing for E1 program effects was completed in Phase 1, potentially influencing TVP EM&V results. Subsequent phases have addressed this by balancing control group selection for E1 participation.

Attachment V: Validation of Mixed Effects Regression The NS Power commercial TVP evaluations (Phase 1 – 3)[25](#page-137-1) has historically provided statistically insignificant results. Findings such as "statistically insignificant" can b...

AI summary The document discusses the evaluation of NS Power's commercial TVP programs, noting that results have been statistically insignificant. This is attributed to low statistical power due to methodological choices and the heterogeneity of commercial customer load profiles compared to residential ones. The analysis highlights the need to re-examine the methodology for consistency.

Attachment V: Validation of Mixed Effects Regression [Control Group Selection)](#page-22-1) relative to residential customers. As such, the methodological choice to use a fixed effects regression model for commercial customers, like reside...

AI summary This section discusses the validation of a mixed effects regression model used in the Phase 4 evaluation to account for the heterogeneous nature of commercial load and baseloads. It contrasts this approach with the fixed effects model, which was deemed unsuitable for commercial customers due to the risk of significant Type II error.

Demand Response Overlap between NS Power and E1 In its Decision on E1's 2026 DSM Extension Application (M12249), the Board provided: The Board agrees that concerns about E1's demand response programs are better addressed in its consultatio...

AI summary The Board's decision on E1's 2026 DSM Extension Application highlights concerns about potential overlap between E1's demand response programs and NS Power's Critical Peak Pricing Program. It emphasizes the need for E1 to address these issues in its upcoming DSM Plan application and expects NS Power to cooperate in exploring these matters.

Attachment VII: Board Directives and Consensus Agreement Commitments 2025/26 Season (M12499), NS Power provided an update on the status of these directives and commitments. In its letter on NS Power's December 2025 Year Four Report extensi...

AI summary The Board has approved NS Power's request to extend the filing date of the TVP 2024/25 EM&V report to June 30, 2026, and has deferred some directives and commitments to future sessions or the EM&V report. NS Power provided an update on the status of these directives as Attachment 1.

Conclusion The Year Five Report ( Appendix A ) provides the second evaluation of the MURB TOU Tariff and continues the Company's assessment of customer response to this time-varying rate. While the evaluation scope was narrower than prior...

AI summary The Year Five Report evaluates the MURB TOU Tariff and customer response to time-varying rates. Despite a narrower scope due to AMI data limitations, the report uses the EM&V framework to assess impacts. NS Power plans to resume the TVP Pilot in 2026/27 under the Board-approved tariff (M12451) and will notify stakeholders 30 days before resuming.

Nova Scotia Power (NS Power, Company) launched the Time-Varying Pricing (TVP) Tariff Pilot on November 1, 2021, with the purpose of encouraging customers to shift load from Winter peak periods to Winter off-peak periods. These tariffs pres...

AI summary Nova Scotia Power launched a Time-Varying Pricing (TVP) Tariff Pilot in 2021 to shift customer load from winter peak to off-peak periods. The fifth annual report covers April 1, 2025, to March 31, 2026. A cybersecurity incident affected data collection, and temporary modifications were applied to TVP Tariffs due to operational constraints. AMR meters were installed for MURB TOU participants, but data collection for the control group was incomplete, affecting evaluation results.

Introduction Time-Varying Pricing (TVP) is a scalable customer-focused demand response (DR) program. In Nova Scotia, the TVP pilot enrolment has grown significantly since it began in November 2021. Since its inception, the TVP pilot has of...

AI summary Time-Varying Pricing (TVP) is a demand response program in Nova Scotia that offers Time-of-Use (TOU) and Critical Peak Pricing (CPP) tariffs. Due to a cyber incident, these tariffs were temporarily paused during the 2025/26 Winter, except for the MURB TOU Tariff, which continued with AMR meters. The evaluation of the program is limited to the MURB TOU Winter-period performance, primarily February and March 2026.

Near Term Next Steps - June 26, 2026 stakeholders provide comments on this deck to NS Power - By June 30, 2026 NS Power to file Phase Four (2024/25) Evaluation Report - July 14, 2026 Technical Session 1 (EM&V) - By July 31, 2026 NS Power t...

AI summary The document outlines key near-term steps in a regulatory proceeding, including stakeholder comments, evaluation reports, technical sessions, and stakeholder engagement activities scheduled from June 2026 through the fall/winter of 2026.

Agenda - Welcome & Introduction - Evaluation, Measurement & Verification (EM&V) Scope - Evaluation Methodology - TVP Pilot Enrollment - Year Four Report Findings - Update on Year Five (2025/26) Evaluation Report - Topics for Discussion at...

AI summary The agenda outlines a series of topics for discussion, including the evaluation and verification of programs, the methodology for evaluation, the TVP pilot enrollment, findings from the Year Four report, an update on the Year Five evaluation report, and near-term next steps.

Cohort - Cohorts are numbered based on the Phase they enrolled in the TVP program. For example, Cohort 4 refers to participants recruited in Phase 4 (i.e. the fourth Winter), which started November 2024. - Notably, the pre-pilot period for...

AI summary The document explains that cohorts in the TVP program are numbered based on the phase they enrolled in, with Cohort 4 representing participants from Phase 4, starting in November 2024. The pre-pilot period for each cohort is static, and the text introduces the EM&V scope for the program.

- This process was further streamlined & improved by including E1 program participation balancing in the control group selection process. Previous Method New Method Projected Benefit Neighbourhood criteria based on feeder and Neighbourhood...

AI summary The process for selecting control groups in program evaluations has been improved by incorporating E1 program participation balancing and Eco Shift participation into the selection process, which is expected to improve load similarity metrics and streamline the process.

Data Aggregation This year, the load impact regression modeling uses hourly AMI data rather than customer-level aggregated AMI data by peak period, which was used in Phases 1, 2, and 3. The use of hourly AMI data resulted in an increase in...

AI summary The load impact regression modeling in Phase 4 uses hourly AMI data instead of aggregated data by peak period, increasing the sample size and reducing the margin of error compared to earlier TVP evaluation reports.

Conclusion of the Review Econoler provided guidance and clarifications to NS Power to ensure a consistent and appropriate application of the overall evaluation methodology established for TVP Phase 1 to 3. Additionally, based on the discus...

AI summary Econoler provided guidance and clarifications to NS Power to ensure consistent application of the evaluation methodology for TVP Phases 1 to 3. Econoler also validated the methodological changes in the draft report for TVP Phase 4, finding the justifications provided by NS Power to be sound.

Technical Session 2 Topics Technical Session 2 is to be scheduled for Fall 2026, and is expected to cover the following: - Recommended EM&V refinements - Review of System Planning related values applicable to TVP - Updated ELCC study resul...

AI summary Technical Session 2, scheduled for Fall 2026, will address EM&V refinements, TVP-related system planning values, ELCC study results, system load impacts, avoided costs application, TVP pricing alignment, and an updated rate design scorecard.

August 7, 2026 Regulatory Counsel Nova Scotia Power Inc. PO Box 910 Halifax NS B3J 2W5 Kathleen Murray [[email protected]](mailto:[email protected]) Dear Ms. Murray: M12932 – Nova Scotia Power Inc. – Time-Varying Pricing...

AI summary Nova Scotia Power Inc. submitted evaluation reports for its Time-Varying Pricing Pilot Program for 2024/25 and 2025/26. The Nova Scotia Energy Board has decided to review both reports concurrently through a paper hearing process, with specific deadlines for interventions and submissions.

Request IR-2: Refer to Exhibit N-1, Year 4 Report Results, Summary of Year Four Evaluation Results, pp. 2-3 of 5, which states: Importantly, the Year Four evaluation identifies statistically significant load shifting for commercial TOU and...

AI summary The Year Four evaluation found statistically significant load shifting for commercial TOU and CPP tariffs, a first-time occurrence, and confirmed residential TVP load reductions during peak periods. The request asks NS Power to explain these findings, including program changes, methodological differences, implications for future evaluations, and the timing of peak load reductions.

Request IR-5: Refer to Exhibit N-2, Year 4 Report Results Appendix A, pages 14-15, (PDF pages 28-29 of 157), Chapter 2 Control Group Selection and section 2.2 Commercial TOU & CPP, which states: The same methodological approach as used wit...

AI summary The document raises questions about the effectiveness of control groups in evaluating the Commercial TOU and CPP programs, the statistical significance of results, and implications for future program design. It also inquires about the status of deliverables in NS Power's 2026/27 Work Plan and the impact of a cyber event on scheduling.