E-22024 DSM Programs Evaluation Reports
406 passages
2024 DSM PROGRAMS EVALUATION REPORTS Final DSM Reports March 27, 2025 In Collaboration with:
AI summary The document announces the submission of final 2024 DSM Programs Evaluation Reports on March 27, 2025, with collaboration indicated through visual logos (image placeholders). No detailed content or findings from the evaluations are provided in the text.
2024 DSM PROGRAMS EVALUATION Final Report
AI summary Final evaluation report assessing the effectiveness of 2024 Demand Side Management (DSM) programs, focusing on program performance, cost recovery, and alignment with regulatory objectives.
DEFINITIONS Accuracy Reflects the proximity of measurements to the true value. Net savings Energy or peak demand savings that can be reliably attributed to a program. This includes effects such as free-ridership and spillover that negative...
AI summary The document defines key terms related to energy efficiency and measurement accuracy, including net savings, net-to-gross ratio, non-sampling error, and peak coincidence factor. These definitions are critical for evaluating the effectiveness of energy programs and ensuring accurate measurement of energy savings.
EfficiencyOne (E1), an independent, non-profit organization, is responsible for helping Nova Scotians improve the energy efficiency of their homes and workplaces by designing, marketing, and delivering demand-side management (DSM) for Nova...
AI summary EfficiencyOne (E1) is an independent non-profit that delivers energy efficiency and demand response programs in Nova Scotia through the Efficiency Nova Scotia franchise. An evaluation of E1's 2024 DSM program portfolio confirmed significant energy and GHG savings, with 172.760 GWh in net electrical energy savings and 81,577 tonnes of CO2 eq in avoided GHG emissions.
1 Evaluation Scopes and Objectives The 2024 Portfolio Evaluation Plan was based on the Evaluation Schedule outlined in the Overall Strategic Evaluation Plan[4](#page-10-1) that provides the framework and approach to guide evaluation planni...
AI summary The 2024 Portfolio Evaluation Plan outlines the approach for evaluating demand-side management (DSM) activities from 2023-2025. It emphasizes prioritizing evaluations based on program savings, uncertainty, changes in design, regulatory requirements, and timely feedback. The plan includes impact, process, and market evaluations, with a focus on comprehensive or condensed impact evaluations.
1.1 Impact Evaluation Objectives and Scope The impact evaluation activities were aimed at determining: - › Gross electrical energy and peak demand savings at the meter and at the generator - › Available demand response (DR) capacity for de...
AI summary The impact evaluation objectives include assessing energy and peak demand savings, demand response capacity, net-to-gross ratios (NTGRs), effective useful life (EUL) of measures, and GHG emissions. Two evaluation types are outlined: comprehensive (reviewing baseline definitions, methodologies, parameters, and NTGRs) and condensed (reusing prior-year parameters). Evaluations occur every three years for most programs, with factors like program maturity and calculation complexity influencing the approach.
Demand-side Management Measure Assessment Document The impact evaluation scope for 2024 also included an update of the Demand-side Management Measure Assessment (DSM MA) document. This update covered all prescriptive measures and their par...
AI summary The 2024 update of the Demand-side Management Measure Assessment (DSM MA) includes prescriptive and semi-prescriptive measures, reviews of parameters like unitary energy savings and peak demand-to-energy ratios, and annual updates for LED lighting and appliance retirements, using new data sources and methodologies.
Net-to-gross Ratio Review The evaluation scope included a NTGR review that served to update the NTGR questionnaires and algorithms for the DSM portfolio so that they were consistent across programs and in line with the latest best practice...
AI summary The NTGR review updated algorithms and questionnaires for E1's DSM programs to align with best practices, focusing on self-report methods. It covered rule alignment, weighting, and survey formulation, applicable to 2024 programs and the 2023-2025 DSM Plan period, excluding Instant Savings due to prior data collection.
1.2 Process and Market Evaluation Objectives and Scopes Two market evaluations were completed in 2024. Market evaluation activities were aimed at achieving the following objectives: - › Identify new opportunities for residential electricia...
AI summary Two market evaluations in 2024 focused on residential electrician-installed Efficient Product Installation (EPI) and BNI lighting for Business Energy Rebates – Instant Rebates (BER-IR). Four process evaluations addressed Affordable Multifamily Housing (AMH), Affordable Single-family Homes (ASFH), BER-IR, and Custom New Construction, aiming to identify barriers, explore expansion opportunities, and update program models.
2 Evaluation Methodology This section presents the methodologies used and the activities carried out to evaluate E1 DSM program components and services through impact, process, and market evaluations.
AI summary This section outlines the methodologies and activities used to evaluate E1 DSM program components and services through impact, process, and market evaluations, focusing on assessing program effectiveness and outcomes.
2.1 Impact Evaluation The impact evaluations were conducted through a range of activities such as tracking sheet audits, datacollection tool development, project reviews assisted by participant follow-up interviews, energy model reviews, o...
AI summary The impact evaluations were conducted through activities like tracking sheet audits, data collection tool development, project reviews with participant interviews, energy model reviews, on-site visits, and analyses. Subsequent subsections outline the steps taken to perform these evaluations.
2.1.1 Tracking Sheet Audits The final tracking sheets submitted to the Evaluator by E1 contained both data for all completed projects for 2024 and the tracked results required to calculate final savings. The final tracking sheets were audi...
AI summary The Evaluator audited final tracking sheets submitted by E1 to assess data consistency and completeness for 2024 program projects. Corrected tracked values were determined to ensure accurate savings calculations, with evaluated savings based on audited results.
2.1.2 Data-collection Tool Development and Sampling Strategy Data-collection tool development and sampling were carried out for Instant Savings, Affordable Multifamily Housing (AMH), Affordable Single-family Homes (ASFH)[,](#page-15-4) 9 E...
AI summary The section outlines the development of data-collection tools for programs like Instant Savings, AMH, ASFH, EPI, and BER. Instruments included surveys, interviews, and protocols informed by documentation reviews and staff interviews, with an integrated approach to process and market evaluation.
Surveys and Interviews This subsection describes the data-collection activities conducted for the impact evaluations. It should be noted that surveys and interviews were often integrated to collect impact, process, and market information a...
AI summary This subsection outlines the data collection methods used in impact evaluations, including participant and non-participant surveys conducted between September and December 2024. Surveys were primarily conducted by telephone, with exceptions for online formats used in Instant Savings, Green Heat, and Home Energy Assessment programs.
Site Visits and Project Reviews with Follow-up Site Visits or Interviews The Evaluator performed a total of 146 project reviews during the summer and fall of 2024, 70 of which were complemented through site visits and 14 were complemented...
AI summary The Evaluator conducted 146 project reviews in 2024, including site visits and phone interviews, to validate installations and gather information on free-ridership and spillover effects for various programs such as Efficient Product Installation, Affordable Multifamily Housing, and Strategic Energy Management.
Energy Model Reviews The Evaluator performed energy model reviews for the 12 Custom New Construction projects to verify the accuracy of energy models. After the initial file reviews, the Evaluator concluded that the available documentation...
AI summary The Evaluator conducted energy model reviews for 12 Custom New Construction projects, verifying model accuracy by comparing them to as-built drawings and project documentation. No site visits were required, and adjusted models were used to establish evaluated savings.
2.1.5 Gross Savings Analysis Gross savings refer to changes in energy consumption resulting from actions taken by participants regardless of their reasons for participating in a program. Upon completion of the impact evaluation activities...
AI summary Gross savings analysis evaluates energy consumption changes from program participation, comparing data collected by the Evaluator with E1's tracked data. It involves compiling measure savings and evaluated parameters to assess program effectiveness.
Net-to-gross Assessment and Net Savings Calculations Free-ridership levels were established for select program components by conducting self-report surveys or in-depth interviews. Those surveys and interviews included a set of questions us...
AI summary The document outlines methods for calculating free-ridership and spillover levels in energy efficiency programs, using surveys and interviews. It details updates for 2024 evaluations, excludes low-income participants due to nil free-ridership, and references NTGR reviews and spillover assessments for various program components.
2.2 Process and Market Evaluations Process and market evaluations were conducted using a range of activities such as program component documentation as well as secondary data reviews, jurisdictional scans, logic model reviews, as well as p...
AI summary Process and market evaluations were conducted through documentation, data reviews, surveys, and interviews, focusing on Efficient Product Installation, Business Energy Rebates – Instant Rebates, Affordable Multifamily Housing, Affordable Single-family Homes, and Custom New Construction programs.
Documentation Review The Evaluator reviewed all relevant evaluation and program component-specific documentation such as program manuals, marketing materials, application forms, tracking sheets, and any other information on changes made to...
AI summary The Evaluator reviewed program documentation, including manuals, marketing materials, and application forms, and conducted annual staff interviews to assess improvements and changes to program components since the last evaluation.
Data-collection Tool Development and Sampling Strategy As described in Subsection [2.1.2](#page-15-3) above, the Evaluator used an integrated approach to developing datacollection tools that serve all evaluation types where possible. For i...
AI summary The Evaluator developed integrated data-collection tools for process, market, and impact evaluations to reduce respondent burden and ensure integrated results. Data sources include Nova Scotia Power and Emera Inc.'s 2023 reports.
Data Collection This subsection describes the data-collection activities conducted for the process and market evaluations. As discussed above, surveys and interviews were integrated where applicable to collect information for the impact, p...
AI summary The data collection process included secondary data analysis, in-depth interviews with stakeholders, and participant surveys targeting New Construction, Affordable Single-family Homes, and Business Energy Rebates – Instant Rebates programs. Interviews provided qualitative insights, while surveys quantified participant perspectives on program components and service quality.
Logic Model Update The Evaluator reviewed the Custom New Construction logic model to assess how well the program logic and theory of change address barriers to achieving program objectives. To do so, the Evaluator reviewed the most recent...
AI summary The Evaluator reviewed E1's Custom New Construction logic model to assess how well its program logic and theory of change address barriers to achieving objectives. The review focused on examining program objectives and barriers through the most recent logic model and theory of change provided by E1.
Analysis The results of the process and market evaluation activities were analyzed in relation to the research objectives identified in Subsection [1.2](#page-13-0) above. The results from all evaluation activities were consolidated and tr...
AI summary The analysis consolidated and triangulated results from process and market evaluation activities, aligning them with research objectives outlined in Subsection 1.2. Findings were validated through a preponderance of evidence from multiple evaluation sources.
3 Impact Evaluation Results This section presents an analysis of the impact evaluation results for all program components by comparing 2024 tracked electrical energy and peak demand savings with evaluated electrical energy and peak demand...
AI summary This section compares 2024 tracked electrical energy and peak demand savings with evaluated savings, presenting NTGRs, lifetime energy savings, and GHG emission reductions as key evaluation outcomes.
Affordable Multifamily Housing - › Despite a slight increase (5%) in participation compared to 2023 levels, AMH achieved lower gross electrical energy and peak demand savings than in 2023. - › In 2024, prescriptive projects generated the m...
AI summary AMH participation rose 5% in 2024 but achieved lower energy and peak demand savings than 2023. Prescriptive projects dominated savings (80% energy, 88% peak demand). Evaluator adjusted savings upward for electrical energy (1.016 ratio) but downward for peak demand (0.003-0.006 MW reductions). Net savings aligned closely with E1 tracking despite adjustments.
Efficient Product Installation - › Compared to 2023 levels, participation in 2024 increased by 2% while average electrical energy savings per participant decreased by 6.3%. This reduction in savings per participant was mainly due to the up...
AI summary Efficient Product Installation (EPI) participation rose 2% in 2024 compared to 2023, but average electrical energy savings per participant fell 6.3% due to 2024-2025 DSM MA updates, particularly reduced smart thermostat and LED lamp savings. Evaluated net electrical energy savings were 6% lower than tracked, while peak demand savings increased 1% due to DSM MA updates and NTGR adjustments.
Green Heat - › Green Heat participation levels decreased by 21% compared to 2023 levels, particularly for MSHPs and demand reduction measures for which participation decreased by 36% and 19% respectively. - › The net evaluated electrical e...
AI summary Green Heat participation dropped 21% from 2023, with MSHPs and demand reduction measures declining 36% and 19% respectively. Net savings evaluations were lower than E1's tracking due to 2024 billing analysis updates on unitary savings for MSHPs and biomass measures.
Home Energy Assessment - › With 5,367 projects, the 2024 participation level was the highest observed since the program component was launched. The average gross electrical energy savings per home slightly decreased compared to 2023 levels...
AI summary The 2024 Home Energy Assessment program achieved record participation with 5,367 projects, driven by the Canada Greener Homes Grant. Energy savings slightly decreased from 2023 but remained high. Updated adjustment ratios from a 2024 billing analysis revealed higher savings than E1's tracked results due to DSM MA updates.
Business Energy Rebates - › In 2024, Instant Rebates participation decreased by 26% compared to 2023 levels. Application Rebates participation increased by 18% in 2024, while gross electrical energy savings per Application Rebates particip...
AI summary In 2024, Business Energy Rebates (BER) saw a 26% drop in Instant Rebates participation but an 18% rise in Application Rebates. Free-ridership for LED products decreased after algorithm updates, while NTGR improvements increased energy savings. Evaluated savings for Application Rebates matched E1's data, but Instant Rebates showed 4-6% higher savings.
Custom - › Compared to 2023, Custom participation decreased in 2024. While Retrofit and Building Optimization participation levels decreased, New Construction participation significantly increased in 2024. - › Following the project reviews...
AI summary Custom program participation decreased in 2024, with Retrofit and Building Optimization participation declining while New Construction increased. The Evaluator adjusted energy and peak demand savings estimates, noting reduced free-ridership (15% for Retrofit, 28% for New Construction) and no spillover. Evaluated savings were 8% and 5% higher than E1's tracked figures.
Strategic Energy Management - › The SEM participation level has remained stable over the past five years. Electrical energy savings per participant were similar to those in 2023. - › The measurement and verification (M&V) methodologies app...
AI summary Strategic Energy Management (SEM) participation has remained stable over five years, with energy savings per participant consistent with 2023 levels. Measurement and verification (M&V) methods were deemed accurate, and peak demand savings estimates improved significantly. Evaluated net energy and peak demand savings aligned closely with initial E1 tracking.
Small Business Energy Solutions - › In 2024, the number of units rebated under Small Business Energy Solutions increased by 53% compared to 2023 levels, with the DIY path remaining the most popular and accounting for 98% of all units rebat...
AI summary SBES saw a 53% increase in rebated units in 2024, with DIY path dominating at 98%. A survey of 30 businesses found no non-participant spillover. Energy savings were 1% and 5% higher than E1's tracked savings due to 2024-2025 DSM MA updates.
Residential Demand Response › Tracked results were not fully compiled, as multiple methodologies for establishing the available DR capacity of the measures included in 2024 were being considered. The Evaluator decided on the final methodol...
AI summary The 2024 evaluation of residential demand response (DR) programs faced delays due to unresolved methodologies for measuring DR capacity. The Evaluator finalized the methodology as part of the 2024 evaluation, resulting in the absence of tracked results or realization rates for the program component.
Table 7: 2024 Free-ridership, Spillover, and NTGRs Program Component and Measure Type Free-ridership Levels Spillover Levels NTGR Residential Appliance Retirement a Refrigerators 43% 0% 0.57 Freezers 45% 0.55 Air Conditioners 47% 0.53 Smal...
AI summary Table 7 presents data on free-ridership, spillover, and net-to-gross ratios (NTGR) for various energy efficiency programs in 2024. It includes metrics for residential and non-residential programs, highlighting levels of free-ridership and spillover across different measures and participant categories.
4.1 Participant and Partner Satisfaction The 2024 evaluation revealed high participant satisfaction with E1 and its programs. Affordable Multifamily Housing, Affordable Single-family Homes, Efficient Product Installation, Custom (New Const...
AI summary The 2024 evaluation showed high participant satisfaction with E1 and its programs, with scores of 7.9 or higher for several initiatives. Program partners were generally satisfied but less so than participants. A table presents the average satisfaction scores for program components on a 0-to-10 scale.
Affordable Multifamily Housing To collect information on participant and distributor perspectives, the Evaluator conducted interviews with program staff, energy auditors (EAs), and participants. The key findings specific to the Affordable...
AI summary The Evaluator gathered perspectives from program staff, energy auditors, and participants to assess the Affordable Multifamily Housing initiative, highlighting key findings from the process evaluation.
› The Evaluator surveyed high levels of satisfaction overall. All 10 interviewed participants said that they were highly satisfied with Affordable Multifamily Housing and "would definitely" recommend the program component to other housing...
AI summary The Evaluator found high satisfaction with the Affordable Multifamily Housing (AMH) program, with all 10 interviewed participants reporting strong approval and a Net Promoter Score of 100. Energy Auditors praised the program's simplicity and upgrade selection, highlighting its ease of implementation and effectiveness.
Affordable Single-family Homes To collect information on program staff, partner, and participant perspectives, the Evaluator conducted surveys with participants as well as interviews with heat pump contractors, program staff, and delivery...
AI summary The evaluation of the Affordable Single-family Homes (ASFH) program involved surveys and interviews with participants, heat pump contractors, program staff, and delivery agents to gather perspectives on program implementation and outcomes.
› Despite overall satisfaction, there are three irritants with the program component. Long wait times (participants, DAs, and heat pump contractors), insufficient program and product information (participants), and reporting processes and...
AI summary The program component faces three irritants: long wait times for participants, DAs, and heat pump contractors; insufficient program/product information for participants; and cumbersome reporting processes with non-user-friendly tools for DAs and heat pump contractors. These issues are highlighted as sources of dissatisfaction.
Business Energy Rebates – Instant Rebates To collect information on participant and distributor perspectives, the Evaluator conducted surveys with participants as well as interviews with distributors. The key finding specific to the Busine...
AI summary The evaluation of Business Energy Rebates – Instant Rebates (BER-IR) reports high satisfaction among participants (9.1/10) and distributors (8.3/10). Participants praised interactions and rebate amounts, while distributors highlighted E1's service support (9.3/10) and rebate processing (8.4/10).
barriers appear to be consistent across both MURB and non-MURB markets. › The share of modelling costs covered by the program's modelling incentive may be insufficient to motivate some builders. Both staff and modellers noted that modellin...
AI summary Barriers in the Custom New Construction program include insufficient modelling incentives, a shortage of energy modellers, and challenges in adapting to NECB 2020. Modelling costs are rising, with 30% of projects hitting caps in 2024. The program needs more modellers and updated logic models to address barriers. Transitioning to NECB 2020 requires builders to adopt higher efficiency technologies and collaborate with modellers.
Table 15: 2024 Recommendations on Residential Program Components No. Recommendation AMH – R1 To further aid understanding, consider providing participants with case study examples of incentive calculations to show pre-tax costs and how inc...
AI summary The document outlines recommendations for improving residential energy efficiency programs in Nova Scotia, including providing case studies, support for participants, improving audit reporting, and tracking wait times and satisfaction levels to assess program performance.
Program Components Bibliographic References Iowa Utilities Commission. Iowa Energy Efficiency Statewide Technical Reference Manual Version 5.0, Volume 3: Nonresidential Measures, July 22, 2020, pp. 43-47. Jesse Remillard and Nick Collins,...
AI summary This document compiles bibliographic references for energy efficiency program components, including technical reference manuals from Iowa, Vermont, Illinois, Minnesota, and other jurisdictions, along with studies on energy use in the cannabis industry and climate data. It emphasizes standardized evaluation methods and regulatory frameworks for energy efficiency initiatives.
Program Components Bibliographic References http://interchange.puc.state.tx.us/WebApp/Interchange/Documents/40891_20_8 71637.PDF. Accessed: Feb 18, 2016. Minnesota Commerce Department, State of Minnesota Technical Reference Manual for Ener...
AI summary The text lists bibliographic references for energy efficiency programs, technical manuals, and evaluations from various states and organizations, including Minnesota, Iowa, Massachusetts, and Pennsylvania, as well as studies on refrigeration equipment and rebate programs.
Adjustment Ratio Margins of Error – Stratified Sample Below is a description of the steps followed to calculate the margins of error for adjustment ratios applied to electrical energy and peak demand savings based on the energy savings exa...
AI summary The document outlines the methodology for calculating margins of error in adjustment ratios for electrical energy and peak demand savings, using data from the 2024 Custom Retrofit evaluation. This process involves a stratified sample approach to assess energy savings accuracy.
Calculation of the Weighted Average of Adjustment Ratios In 2024, different types of projects were reviewed for Custom Retrofit; the example herein is for regular retrofit projects. For these projects, the Evaluator used a stratified sampl...
AI summary The document details the calculation of a weighted average adjustment ratio (AR) for retrofit projects in 2024. Using a stratified sample of 18 projects, the Evaluator applied a formula involving stratum weights and energy savings, resulting in a weighted average AR of 1.021. This ratio reflects adjustments to energy savings estimates across different project strata.
Calculation of the Margins of Error The margin of error on the adjustment ratio of Custom Retrofit energy savings was established by using the following formula, which compares the error in evaluated savings divided by the evaluated saving...
AI summary The document explains the calculation of the margin of error for adjustment ratios (ARs) in Custom Retrofit energy savings, resulting in a 4.9% margin. In 2024, this calculation was applied to Custom and BNI DR, while smaller programs like SEM used a census approach.
RESIDENTIAL EFFICIENT PRODUCT REBATES PROGRAM Final Report 2024 DSM EVALUATION March 25, 2025 In collaboration with:
AI summary The document presents the Final Report of the 2024 DSM Evaluation for Nova Scotia's Residential Efficient Product Rebates Program, dated March 25, 2025. It highlights collaboration with unnamed entities, though specific details about program outcomes, stakeholder input, or evaluation findings are not provided in the text snippet.
Evaluation Approach The 2024 evaluation was aimed at calculating program component gross and net results, namely electrical first-year and lifetime energy savings, peak demand savings, as well as avoided greenhouse gas (GHG) emissions. [Ta...
AI summary The 2024 evaluation aimed to calculate program component gross and net results, including electrical first-year and lifetime energy savings, peak demand savings, and avoided greenhouse gas emissions. Table 1 summarizes the types of evaluation conducted for each program component and the corresponding methodology.
Table 1: Summary of 2024 Residential Efficient Product Rebates Program Evaluation Program Evaluation Type Component Impact Process Market Methodology Appliance Retirement Condensed › Tracking sheet audit › Unitary savings review › Calculat...
AI summary The 2024 Residential Efficient Product Rebates Program Evaluation includes assessments of the Appliance Retirement and Instant Savings components. The evaluation uses methods such as tracking sheet audits, unitary savings reviews, and GHG emission reduction calculations. The Instant Savings component also includes participant surveys and market effects analysis.
ARet Findings and Recommendations This subsection presents the key findings from the 2024 ARet evaluation. The Evaluator has no specific recommendation for ARet. 2024 ARet-Finding: ARet achieved both its net electrical energy savings and p...
AI summary The 2024 ARet program met its energy and peak demand savings targets by 22% and 18%, respectively, with a 39% increase in participation driven mainly by refrigerator and freezer retirements. Savings tracked by the Evaluator were slightly lower than those by E1.
Instant Savings Findings and Recommendations This subsection presents the key findings from 2024 Instant Savings evaluation. The Evaluator has no specific recommendation for Instant Savings. 2024 Instant Savings-Finding: Instant Savings su...
AI summary 2024 Instant Savings exceeded energy savings (77%) and peak demand savings (45%) targets. Participation rose 82% due to E1's campaign, with LED lighting driving most savings. Non-lighting savings increased, free-ridership dropped, and evaluator-estimated savings outpaced E1's tracked results.
Program Tracked and Evaluated Savings [Table](#page-85-0) 3 below compares E1 tracked energy and peak demand savings to Evaluated savings at the generator. The realization rate and NTGR are also presented and correspond to the rounded aver...
AI summary The table compares E1 tracked energy and peak demand savings to evaluated savings at the generator, showing realization rates and NTGR values, which are calculated as the ratio of net savings to gross savings.
INTRODUCTION EfficiencyOne (E1), an independent, non-profit organization, is responsible for helping Nova Scotians improve the energy efficiency of their homes and workplaces by designing, marketing, and delivering demand-side management (...
AI summary EfficiencyOne (E1), a non-profit organization, manages demand-side management (DSM) programs in Nova Scotia through the Efficiency Nova Scotia (ENS) franchise. E1's 2024 DSM program portfolio includes the Residential Efficient Product Rebates program with components like Appliance Retirement and Instant Savings. Econoler was commissioned to evaluate these programs, focusing on baseline definitions, savings calculations, and net-to-gross ratios.
1.2 Follow-up on Past Evaluation Report Recommendations No recommendations were made for ARet in the 2023 evaluation.
AI summary No recommendations were made for Appliance Retirement (ARet) in the 2023 evaluation report, indicating that the program's performance or outcomes did not necessitate corrective actions or improvements.
Table 5: 2024 ARet Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › What are the evaluated first-year and lifetime g...
AI summary Table 5 outlines the 2024 ARet Evaluation Approach, focusing on calculating gross and net results through methods like tracking sheet audits and unitary savings reviews. It addresses research questions on data accuracy, energy savings, and greenhouse gas emission reductions.
Unitary Savings Review As part of a major update to the 2024-2025 Demand-side Management Measure Assessment (DSM MA)[,](#page-90-2) 4 a unitary savings review was conducted for all measures. The unitary savings review entailed a literature...
AI summary A unitary savings review was conducted for the 2024-2025 Demand-side Management Measure Assessment (DSM MA), involving a literature review of technical manuals, metering studies, and program evaluations to refine methodologies for calculating energy and peak demand savings. The review specifically examined refrigerators and freezers, considering changes in manufacturing year-classes and unit sizes.
3 ARet Impact Evaluation This section presents the findings of the 2024 condensed impact evaluation aimed at determining gross and net electrical energy and peak demand savings.
AI summary This section outlines the 2024 condensed impact evaluation of ARet, focusing on quantifying gross and net electrical energy savings and peak demand reductions. The evaluation assesses the program's effectiveness in achieving energy efficiency goals.
3.1 Tracking Sheet Audit To ensure program results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1 as well as correcting the track...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, correcting tracked savings as necessary. Corrected results are detailed in Appendix I, ensuring reliable program outcome reporting.
4 ARet Key Findings and Recommendations The 2024 ARet evaluation included a condensed impact evaluation, and its main objectives were as follows: › Calculate gross and net ARet results, namely electrical first-year and lifetime energy savi...
AI summary The 2024 ARet evaluation achieved 2.365 GWh in net electrical energy savings and 0.337 MW in peak demand savings, exceeding targets by 22% and 18% respectively. Participation rose 39% YoY, driven by refrigerator/freezer retirements. Savings were 1% lower than E1's tracked values, but overall program goals were met.
6 Instant Savings Evaluation Approach The 2024 Instant Savings evaluation included a comprehensive impact evaluation whereby NTGRs, namely free-ridership, as well as unitary savings were reviewed and updated. The main objectives of the 202...
AI summary The 2024 Instant Savings evaluation aimed to calculate gross and net savings, including energy and peak demand savings, and avoided GHG emissions. The evaluation reviewed free-ridership and unitary savings, with research questions and methods outlined in a table.
Table 18: 2024 Instant Savings Evaluation Approach Evaluation Objective Research Question Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › What are the unitary savings and EUL v...
AI summary This section outlines the methodology for evaluating the 2024 Instant Savings program, focusing on calculating both gross and net results. It includes an audit of tracking sheets, unitary savings review, EUL updates, participant surveys, and calculations for energy savings and GHG emission reductions.
Tracking Sheet Audit The Evaluator performed an audit of the final 2024 tracking sheet to ensure it was complete and the entered data were consistent. The results obtained are presented in Appendix V.
AI summary The Evaluator audited the 2024 tracking sheet to ensure completeness and data consistency, with results detailed in Appendix V.
Unitary Savings Review A unitary savings review was conducted for all measures as part of a major update to the 2024-2025 DSM Measure Assessment (DSM MA). [15](#page-109-0) The unitary savings review consisted of a literature review of tec...
AI summary A unitary savings review was conducted for all measures as part of updating the 2024-2025 DSM Measure Assessment. The review analyzed technical references, metering studies, and program evaluations to refine methodologies for calculating energy and peak demand savings, focusing on baseline assumptions, efficiency levels, and interactive effects factors.
Calculations Using Evaluation Results Building on all the above methods and collected data, the Evaluator calculated first-year and lifetime energy and peak demand savings as per the calculation methodology presented in Section [7](#page-1...
AI summary The Evaluator calculated first-year and lifetime energy and peak demand savings using a methodology outlined in Section 7, building on prior data collection and analysis methods.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations, focusing on free-ridership levels in the Instant Savings program. Confidence levels exclude non-sampling errors. The 2024-2025 DSM MA document provides parameters for calculating energy savings and effective useful life values for E1's DSM program measures.
7 Instant Savings Impact Evaluation The objectives of the 2024 Instant Savings impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as effective useful life (E...
AI summary The 2024 Instant Savings impact evaluation aimed to quantify gross/net energy and peak demand savings, annual GHG emissions avoided, effective useful life (EUL) values, and lifetime energy savings from the program.
7.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification and...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of program results. Corrective actions are detailed in Appendix V, leading to corrected tracked savings reported in the document.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to verify data completeness and accuracy in EfficiencyOne's (E1) submissions, ensuring consistency in parameters and calculation steps used to determine program results.
Table 1: Overview of Data Collection Activity Descriptor This Instrument Instrument Type Participant Intercept Survey with Online Option Estimated Time to Complete 5 min Target Audience Instant Savings participants who have purchased LED n...
AI summary This document outlines the data collection activities for a survey targeting participants of the Instant Savings program who purchased LED bulbs and fixtures during the fall 2024 campaign. The survey aims to assess free-ridership, cross-influence, and participant perspectives on program awareness.
H. Cross-Influence ROTATE [(H1+](#page-151-1) [H2-](#page-151-2)[H3)](#page-151-3) AND [(H4](#page-151-4) + [H5-](#page-151-5)[H6)](#page-151-6) SEQUENCES; SHOW [H1](#page-151-1) TO [H3](#page-151-3) ON SAME SCREEN AND H4 TO [H6](#page-151...
AI summary The section includes survey questions assessing how prior participation in Efficiency Nova Scotia programs and exposure to promotional materials influenced customers' decisions to purchase LED lighting products.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - 1. Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitte...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator, aimed at verifying data completeness and accuracy in program results calculations, including gross and net energy and peak demand savings.
Table 2: ME Algorithm - LED bulbs Total Non-A Type Sales Non-A type LED bulbs sold from January – December 2024(extrapolated from retailer data and responses in interview) #LEDJan-Dec LED Sales During Instant Savings' Campaigns LED Bulb Sa...
AI summary The text presents a table detailing the calculation of final market effects for LED bulb sales, including the influence of Efficiency Nova Scotia's programs and the determination of free-ridership levels based on customer awareness of discounts.
Table 1: Summary of 2024 Existing Residential Program Evaluation Program Evaluation Type Methodology Component Impact Process Market AMH Comprehensive X › Program staff interviews › Energy auditor (EA) interviews › Participant, dropped-out...
AI summary The document provides a summary of the 2024 evaluation of existing residential programs, detailing the evaluation types, methodologies, and components for various programs such as AMH, ASFH, EPI, Green Heat, HEA, MHEEP, and Residential Behaviour. The evaluation includes interviews, tracking sheet audits, billing analysis, and GHG emission reduction calculations.
AMH Findings and Recommendations This subsection presents the key findings and recommendations from the 2024 AMH evaluation. 2024 AMH-Finding: AMH generated high levels of satisfaction overall. Interviewed participants were highly satisfie...
AI summary The 2024 AMH evaluation found high participant satisfaction but noted confusion over incentive tax calculations and a need for project management support. Recommendations include case studies for clarity, technical guides, and audit template improvements. EAs expressed dissatisfaction with audit reporting templates, prompting suggestions for streamlining.
ASFH Findings and Recommendations This subsection presents the key findings and recommendations from the ASFH evaluation in relation to the above objectives. 2024 ASFH-Finding: The energy efficient retrofits installed at no cost through AS...
AI summary Participants in the ASFH program report increased home comfort and heating cost savings, but face long wait times and confusion due to limited program knowledge. E1 acknowledges wait time issues and is expanding staff to address backlogs. Recommendations include tracking wait times and improving participant information.
Green Heat Findings and Recommendations 2024 Green Heat - Finding: Green Heat fell short of its net savings targets, achieving 24% and 55% of its electrical energy and peak demand targets respectively. 2024 Green Heat-Finding: Green Heat p...
AI summary The 2024 Green Heat program underperformed, achieving only 24% and 55% of its energy and peak demand targets. Participation dropped due to the CGH Grant's launch, with low savings rates (35% and 71%) for MSHPs and biomass measures. Nil electrical savings were found for some households, and a recommendation to remove wood/pellet fireplace inserts is proposed.
HEA Findings and Recommendations 2024 HEA-Finding: HEA net electrical energy savings exceeded the target of 19.012 GWh by 68%, and the planned net peak demand savings of 4.799 MW by 81%. 2024 HEA-Finding: With 5,367 projects, the 2024 part...
AI summary The 2024 HEA exceeded energy savings targets by 68% and peak demand savings by 81%, with record participation. The CGH Grant's closure may reduce future savings. Discrepancies in savings calculations were addressed, and low-saving wood/pellet fireplaces are recommended for removal from HEA offers.
MHEEP Findings and Recommendations This subsection presents the key findings from the MHEEP evaluation. The Evaluator has no specific recommendation for MHEEP. 2024 MHEEP-Finding: MHEEP net electrical energy savings fell short of targets b...
AI summary The MHEEP program underperformed in 2024, achieving 35% less net energy savings than targets but exceeding peak demand savings by 189%. Participation increased by 19%, yet energy savings per participant fell by 41%. Discrepancies with E1's data stemmed from revised adjustment ratios and heat pump peak demand metrics.
Residential Behaviour Findings and Recommendations This subsection presents the key findings from the 2024 Residential Behaviour evaluation. The Evaluator has no specific recommendation for Residential Behaviour. 2024 Residential Behaviour...
AI summary The 2024 Residential Behaviour program achieved 6.270 GWh in energy savings, below its 8.000 GWh target, but shows promise as it scales. Attrition rates reached 5.2-7.0% among participants, and treatment group customers showed higher engagement in other programs compared to controls.
Program Tracked and Evaluated Savings [Table](#page-3-0) 3 below compares E1 tracked electrical energy and peak demand savings to evaluated savings at the generator. It also includes the realization rate, representing the ratio of evaluate...
AI summary The table compares tracked and evaluated savings from E1 programs, including realization rates and NTGRs. Residential Behaviour lacks a tracking sheet due to random participant selection. NTGR values differ from other sections as they are rounded averages of net savings divided by gross savings.
Table 4: Types of Evaluations Conducted for Each Program Component, 2024 2024 Program Program Component Process Market Impact Existing Residential AMH X Comprehensive ASFH X Condensed EPI X Comprehensive Green Heat Comprehensive HEA Compre...
AI summary Table 4 outlines the types of evaluations conducted for each program component in 2024, including details on market and impact assessments. The Evaluator prepared a DSM evaluation report that includes findings on energy savings, peak demand savings, and GHG emissions for each component.
1 AMH Overview This section describes Affordable Multifamily Housing (AMH), follows up on past evaluation recommendations, and provides an overview of AMH participation history.
AI summary This section outlines Affordable Multifamily Housing (AMH), addresses past evaluation recommendations, and summarizes participation history. It focuses on program oversight and implementation progress for AMH initiatives in Nova Scotia.
1.2 Follow-up on Past Evaluation Report Recommendations No recommendations were made for AMH in the 2023 evaluation.
AI summary The 2023 evaluation report did not provide recommendations for Affordable Multifamily Housing (AMH), indicating no follow-up actions were required based on previous evaluations.
2 AMH Evaluation Approach The 2024 AMH evaluation comprised a comprehensive impact evaluation as well as a process evaluation. The main objectives of the 2024 AMH evaluation were as follows: - › Collect information on participant and EA pe...
AI summary The 2024 AMH evaluation involved both impact and process evaluations, with objectives including collecting participant and EA perspectives and calculating energy savings and GHG emissions. Research questions, methods, and sample sizes were outlined in a table.
Table 5: 2024 AMH Evaluation Approach Evaluation Objectives Research Questions Methodology Collect information on program staff, participant, and EA perspectives › How did participants become aware of the program component? › What is the l...
AI summary Table 5 outlines the 2024 evaluation approach for the Affordable Multifamily Housing (AMH) program, focusing on collecting perspectives from program staff, participants, and Energy Auditors (EAs). It includes research questions about participant awareness, motivations, satisfaction, challenges, and recommendations, with methodology involving interviews.
Dropped-out Participant Interviews In October and November 2024, Narrative Research conducted six telephone interviews among participants who started the program but did not complete the participation process (hereafter referred to as drop...
AI summary In October-November 2024, Narrative Research conducted six interviews with participants who dropped out of the AMH program, focusing on motivations and barriers. Four had MURBs, two had non-bed facilities, and two prescriptive path interviews were partially completed. The analysis aimed to understand dropout reasons and participation challenges.
Non-participant Interviews In October and November 2024, Narrative Research conducted 10 telephone interviews with customers who had contacted E1 to obtain information about the program component but decided not to take part. Six participa...
AI summary Narrative Research conducted 10 interviews in 2024 with E1 program non-participants to identify barriers to participation. Participants included those from comprehensive/prescriptive paths and various housing types (MURBs, rentals, rooming houses). The study aimed to assess initial interest and obstacles to engagement.
Desk Reviews with Participant Interviews In October and November 2024, the Evaluator reviewed the documentation of 10 prescriptive projects, and LMMW Group Ltd performed simulation model reviews for five comprehensive projects. For compreh...
AI summary In October and November 2024, the Evaluator and LMMW Group Ltd. conducted desk reviews and simulation model reviews for 10 prescriptive and 5 comprehensive projects. The reviews aimed to validate the consistency of savings tracked by E1 with project data and simulation outputs, following a protocol detailed in Appendix V.
Calculations Using Evaluation Results Building on all the above methods and collected data, the Evaluator calculated the first-year and lifetime energy and peak demand savings as per the calculation methodology presented in Subsection [4.2...
AI summary The Evaluator calculated first-year and lifetime energy and peak demand savings using collected data and the methodology outlined in Subsection 4.2, which forms the basis for assessing program effectiveness and energy efficiency outcomes.
Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were condu...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations, focusing on random sampling errors. This applies to the 2024 AMH evaluation and DSM MA document, which outline parameters for calculating energy savings and effective useful life values. Nonsampling errors are excluded from confidence level calculations.
3.2.2 Reasons for Initial Interest Among Dropped-out Participants and Non-participants As seen above among participants, when it comes to why dropped-out participants and non-participants were initially interested in AMH, cost-related fact...
AI summary Dropped-out participants and non-participants in AMH were primarily motivated by rebates/financial support, followed by energy efficiency and cost savings. Expectations were vague, with most anticipating financial incentives but unclear on specifics. Negative past rebate experiences tempered some expectations.
Net Promoter Score As illustrated i[n Figure](#page-20-0) 11 below, the Net Promoter Score (NPS) is a tool that measures customer experience with and overall perceptions about a brand or program. It is calculated by subtracting the "Promot...
AI summary The Net Promoter Score (NPS) measures customer satisfaction with programs, calculated by subtracting Detractors from Promoters. All ten Affordable Multifamily Housing (AMH) participants were Promoters, achieving the highest possible NPS of 100, indicating strong program approval.
4 AMH Impact Evaluation The objectives of the 2024 AMH impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as the EUL and associated lifetime energy savings.
AI summary The 2024 AMH impact evaluation aimed to assess gross and net electrical energy and peak demand savings, annually avoided GHG emissions, and the Effective Useful Life (EUL) of AMH programs alongside their lifetime energy savings.
4.1 Tracking Sheet Audit To ensure program results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1 as well as correcting the track...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1 (EfficiencyOne), resulting in corrected tracked savings detailed in Appendix I.
4.2 Gross Savings Gross savings correspond to the change in energy consumption resulting from various electrical energy saving upgrades such as building envelope measures, space heating measures, and domestic hot water (DHW) measures imple...
AI summary Gross savings reflect energy consumption reductions from electrical upgrades by AMH participants, including building envelope, heating, and DHW measures. Methodology for evaluating 2024 AMH gross savings includes assessing interactive effects and EUL values.
4.2.1 Energy Savings For the 2024 AMH evaluation, the Evaluator conducted desk reviews of five comprehensive projects and 10 prescriptive projects. The reviews served to establish the evaluated savings and adjustment ratios if possible.
AI summary The 2024 AMH evaluation involved desk reviews of 15 projects (5 comprehensive, 10 prescriptive) to assess energy savings and determine adjustment ratios. This process aimed to quantify program effectiveness and inform energy efficiency outcomes for affordable multifamily housing.
Comprehensive Path Project Desk Review Findings Through the 2024 comprehensive path project desk reviews, the Evaluator made three types of adjustments: - › The impacts of some measures on other systems, which correspond to interactive eff...
AI summary The 2024 Comprehensive Path Project desk reviews identified modeling errors, unimplemented measures, and parameter discrepancies across five projects. Adjustments reduced energy savings estimates by up to 25%, leading to an overall 0.873 adjustment ratio. Due to a 18% margin of error, non-reviewed projects used tracked savings instead of extrapolated ratios.
Prescriptive Project Review Findings Similarly to energy savings, peak demand savings for AMH prescriptive projects are calculated using the 2024-2025 DSM MA equations. For lighting projects, project reviews revealed that E1 used a peak co...
AI summary The review found that peak demand savings for AMH projects used 2024-2025 DSM MA equations. Lighting projects had a 32% reduction in peak savings after adjusting coincidence factors, but margin of error limited extrapolation. Heat pump projects had no adjustments, leading to equal evaluated and tracked savings. No adjustment ratios were extrapolated for non-reviewed projects.
4.2.3 Interactive Effects In a home, interactive effects occur when the implementation of energy efficiency measures has an impact on the energy consumption of other factors such as heating and cooling. The interactive effects of space hea...
AI summary Interactive effects in energy efficiency measures, such as heating and cooling, are addressed in savings calculations for comprehensive projects. The 2024-2025 DSM MA includes these effects for prescriptive projects. Evaluators assumed lamps operating 10+ hours daily coincided with peak hours, using a peak coincidence factor of 1, while others used annual operating hours divided by total yearly hours.
4.3.1 Evaluated Net Savings Net savings are defined as the changes in energy use that are specifically attributable to AMH. Since spillover and free-ridership effects were considered nil, the net energy savings are equal to the gross savin...
AI summary The section defines net savings for AMH (Affordable Multifamily Housing) as gross savings, assuming no spillover or free-ridership effects. However, AMH missed its 2024 electrical energy and peak demand savings targets by 39% and 64%, respectively, as shown in Figure 14.
5 AMH Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 AMH evaluation were as follows: - › Collect information on participant and distributor perspectives - › Calculate AMH gross and net results, na...
AI summary The 2024 AMH evaluation aimed to gather perspectives from participants and distributors and calculate energy savings, peak demand reductions, and GHG emission avoidance. This section outlines the Evaluator's key findings and recommendations related to these objectives.
2024 AMH-Finding: In 2024, despite a slight increase in participation compared to 2023, AMH achieved lower savings than in 2023. AMH reached its highest participation rate since inception with 83 completed projects generating electrical sa...
AI summary In 2024, AMH achieved lower energy and peak demand savings compared to 2023 despite an 83-project completion rate (5% higher than 2023). Prescriptive projects showed 31% and 29% declines in energy and demand savings, while comprehensive projects saw 85% and 89% increases, but overall savings remained 21% and 23% lower than 2023.
2024 AMH-Finding: Following project reviews, the Evaluator made an overall upward adjustment to electrical energy savings and a downward adjustment to peak demand savings. The 2024 AMH project reviews led to two adjustments for electrical...
AI summary The 2024 AMH project reviews resulted in an upward adjustment to electrical energy savings via a 1.016 AR for prescriptive heat pump projects due to HSPF2 factor changes, and a downward adjustment to peak demand savings from revised peak coincidence factors and comprehensive project modelling errors. Adjustments largely offset each other, yielding negligible net differences.
2024 AMH-Finding: The evaluated net electrical energy savings and peak demand savings were nearly the same as the savings tracked by E1. The net evaluated electrical energy savings were nearly the same as the energy savings tracked by E1 a...
AI summary The evaluated net electrical energy savings and peak demand savings for the 2024 AMH program were nearly identical to E1's tracked savings, with a 2% lower peak demand difference attributed to evaluator adjustments following project reviews.
6 ASFH Overview This section describes the Affordable Single-family Homes (ASFH) program component, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Affordable Single-family Homes (ASFH) program, addresses past evaluation recommendations, and summarizes participation history. It focuses on program oversight and historical engagement data.
6.2 Follow-up on Past Evaluation Report Recommendations No recommendations were made for ASFH in the 2023 evaluation.
AI summary No recommendations were made for Affordable Single-family Homes (ASFH) in the 2023 evaluation report, as noted in the follow-up section.
6.3 Participation History For ASFH, participation is defined as participants (homes) that have completed a project and have positive electrical energy savings, participants that completed a project with no savings, and all participants tha...
AI summary The ASFH program saw a significant increase in participation from 2023 to 2024, with 1,210 homes participating. However, average savings per participant decreased by 48% due to updated adjustment ratios and a shift in project types, including more smart thermostats and non-modelled heat pumps. 43 appliances were replaced, and 17 homes had no energy savings.
7 ASFH Evaluation Approach The 2024 evaluation consisted of a condensed impact evaluation and a process evaluation. The main objectives of the 2024 ASFH evaluation were as follows: - › Collect information on program staff, partner, and par...
AI summary The 2024 evaluation of the Affordable Single-family Homes (ASFH) program focused on collecting perspectives from program staff, partners, and participants, as well as calculating gross and net energy savings, peak demand savings, and avoided GHG emissions. The evaluation included both impact and process components, with key research questions, methods, and sample sizes outlined in a table.
Table 14: 2024 ASFH Evaluation Approach Evaluation Objectives Research Questions Methodology Collect information on program staff, partner, and participant perspectives › How did participants become aware of ASFH? › What are the participan...
AI summary This chunk outlines the evaluation approach for the 2024 Affordable Single-family Homes (ASFH) program, focusing on collecting participant perspectives and calculating gross and net results. It includes survey and interview methods for program evaluation and outlines the use of tools like HOT2000 and the net-to-gross ratio (NTGR) for calculating savings and GHG emission reductions.
Interviews with DAs In October and November 2024, Econoler conducted four interviews with DAs involved in ASFH to obtain their views on the program component, the effectiveness of delivery, and their suggestions for improvements. The guide...
AI summary Econoler conducted four interviews with Delivery Agents (DAs) involved in Affordable Single-family Homes (ASFH) programs in 2024 to assess program effectiveness, delivery challenges, and improvement suggestions. The interview guide is detailed in Appendix X.
Heat Pump Contractors Heat pump contractors began their direct involvement in the heat pump only path of the program component in January 2023. Overall, 43 contractors support heat pump only participants. According to E1, this program comp...
AI summary Heat pump contractors started participating in the heat pump-only program path in January 2023, with 43 contractors supporting ASFH. E1 notes challenges in ramping up due to high demand and varying contractor knowledge. The analysis covers motivations, satisfaction, challenges, and program improvements.
Satisfaction With ASFH In terms of overall satisfaction with ASFH, modelled participants generally appear highly satisfied on average (8.9), with over three-quarters (77%) providing a rating of 8, 9, or 10 out of 10 (se[e Figure](#page-50-...
AI summary Participants in the ASFH program report high satisfaction (8.9/10) with enrollment, installation quality, and product quality, but lower satisfaction with information provided and wait times, particularly between enrollment and home energy assessments. E1 staff note challenges managing application volume and contractor capacity but report progress in reducing backlogs through additional staffing and communication.
Satisfaction Overall, most interviewed DAs are satisfied with the program (7.5 on average), as illustrated in the figure below. The participation levels (9.8) as well as the quality (9.3) of the upgrades are the program elements with the h...
AI summary Most Delivery Agents (DAs) report satisfaction with the ASFH program (7.5 average), particularly its participation levels and upgrade quality. However, dissatisfaction arises from cumbersome data/reporting processes, non-user-friendly software (CIS/SharePoint), and lack of marketing tools. DAs criticize repetitive data entry, platform incompatibility, and poor communication from E1.
Satisfaction With ASFH In terms of overall satisfaction with ASFH, heat pump only participants appear highly satisfied on average (9.4), with over nine in 10 (93%) participants providing a rating of 8, 9, or 10 out of 10. As illustrated in...
AI summary Participants in the Affordable Single-family Homes (ASFH) program, particularly heat pump recipients, report high overall satisfaction (9.4/10), with 93% rating 8-10. Satisfaction is highest for installation quality (97%) and product quality (90%), but lower for information about program steps (67%) and product usage (10% dissatisfied). Contractors and service providers also receive strong approval.
Satisfaction Overall, most interviewed heat pump contractors are generally satisfied with the program (7.0 on average) , as illustrated in the [Figure](#page-61-0) 34 below. The selection of eligible heat pumps (9.0) and the quality of the...
AI summary Heat pump contractors report average satisfaction (7.0) with the ASFH program, citing eligible heat pump selection (9.0), product quality (8.2), and EPP benefits (9.3) as key positives. Dissatisfaction arises from data/record-keeping challenges (5.4), software usability (5.2), delayed E1 responses (n=5), and limited marketing tools (n=2). Contractors find ASFH delivery complex but acknowledge its value.
9 ASFH Impact Evaluation The objectives of the 2024 ASFH impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as the effective useful life (EUL) values and ass...
AI summary The 2024 ASFH impact evaluation aims to assess electrical energy and peak demand savings, annual GHG emissions reductions, and effective useful life (EUL) values for Affordable Single-family Homes, alongside their lifetime energy savings.
9.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1 as well as correcting...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, correcting tracked savings as needed. The results, detailed in Appendix XI, ensure program component results are reliably compiled.
9.3.1 Evaluated Net Savings Net savings are defined as the changes in energy use that are specifically attributable to ASFH. Since spillover and free-ridership effects were considered nil, the net energy savings are equal to the gross savi...
AI summary Net savings for ASFH are equivalent to gross savings due to nil spillover and free-ridership effects. ASFH exceeded 2024 electrical energy savings targets by 32% and peak demand targets by 100%, as illustrated in Figure 35.
10 ASFH Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 ASFH evaluation were as follows: - › Collect information on program staff, partner, and participant perspectives - › Calculate gross and net...
AI summary The 2024 ASFH evaluation aimed to gather perspectives on program implementation and quantify energy savings, peak demand reductions, and GHG emission avoidance. This section outlines the Evaluator's findings and recommendations related to these objectives for Affordable Single-family Homes (ASFH).
2024 ASFH-Finding: Despite overall satisfaction, there are three irritants with the program component. Long wait times (participants, DAs, and heat pump contractors), insufficient program and product information (participants), and reporti...
AI summary The 2024 ASFH program faces three irritants: long wait times for participants, DAs, and contractors; insufficient program/product information for participants; and inadequate reporting tools/software for DAs and contractors. These issues contribute to dissatisfaction despite overall program satisfaction.
2024 ASFH-Finding: Participants would like more information to be provided about the program steps. Limited participant knowledge about ASFH can cause participant confusion and frustration during the initial DA/contractor visit, particular...
AI summary Participants expressed confusion about ASFH program steps, particularly when considering multiple programs like OHPA. Recommendations include improving communication about eligible measures, clarifying application processes, and reinforcing the role of Energy Solution Advisors to enhance participant understanding and reduce frustration.
2024 ASFH-Finding: Record keeping/reporting is time consuming. Having to use two platforms for filling out the required record keeping/reporting forms (CIS) and uploading documentation (SharePoint) is said to lead to repetitive data entry...
AI summary The 2024 ASFH-Finding highlights that record keeping/reporting for Affordable Single-family Homes is time-consuming due to the use of two platforms (CIS and SharePoint), leading to repetitive data entry and unclear requirements. DAs and contractors face challenges with outdated training materials and lack of formal communication on process changes, necessitating streamlined procedures and updated training.
2024 ASFH-Finding: In 2024, 1,210 homes participated in ASFH, nearly five times the number recorded in 2023. Among the 1,210 ASFH participants, 880 implemented building envelope measures, with 696 of them also installing a heat pump. Addit...
AI summary In 2024, 1,210 homes participated in ASFH, a fivefold increase from 2023. Participants implemented building envelope measures, heat pumps, and smart thermostats. While gross energy savings rose 158% and peak demand savings 233%, average savings per participant fell 48% due to updated adjustment ratios and a shift toward projects with lower impact, like smart thermostats and non-modelled heat pumps.
2024 ASFH-Finding: The evaluated net energy and peak demand savings determined by the Evaluator were lower than the initial net savings tracked by E1. The energy and peak demand savings determined by the Evaluator were lower by 21% and 3%...
AI summary The Evaluator found net energy and peak demand savings were 21% and 3% lower than E1's initial tracking, primarily due to revised adjustment ratios. This discrepancy impacts the assessment of program effectiveness for Affordable Single-family Homes (ASFH).
11.2 Follow-up on Past Evaluation Report Recommendations No recommendations were made for EPI in the 2023 evaluation.
AI summary No recommendations were made for Efficient Product Installation (EPI) in the 2023 evaluation report, indicating that the program's performance or outcomes did not require corrective actions or improvements based on the assessment.
11.3 Participation History As presented in [Figure](#page-78-0) 36 below, EPI had 9,993 DSM participants, which represents a 2.4% increase in participation compared to 2023.[28](#page-77-2) In 2024, 150,722 efficient products were installe...
AI summary EPI (Efficient Product Installation) reported 9,993 DSM participants in 2024, a 2.4% increase from 2023. However, efficient product installations decreased by 2%, with LED lamps remaining the top product type (74% of installs). Average savings per participant fell 6.3%, and gross energy savings in 2024 were slightly lower than 2023 despite higher participation.
Table 26: 2024 EPI Evaluation Approach Evaluation Objectives Research Questions Methodology Collect information on participant perspectives › How do participants become aware of EPI? › What is the level of satisfaction with the program com...
AI summary This section outlines the 2024 EPI Evaluation Approach, focusing on participant perspectives, gross and net results calculation, and identifying new opportunities for electrician-installed measures. The evaluation includes surveys, tracking sheet audits, site visits, and jurisdictional scans.
Unitary Savings Review The Evaluator did a complete review and update of unitary savings values and produced the 2024-2025 Demand-side Management Measure Assessment (DSM MA).[31](#page-80-0) The unitary savings review entailed a literature...
AI summary The Evaluator conducted a comprehensive review and update of unitary savings values for the 2024-2025 Demand-side Management Measure Assessment (DSM MA), evaluating technical references and methodologies to refine calculations for energy and peak demand savings, including baseline assumptions and efficiency parameters.
GHG Emission Reduction Calculations To obtain net avoided GHG emissions in CO2 eq for EPI, the Evaluator multiplied net energy savings by the latest Nova Scotia-specific factor for GHG emissions generated by electricity production. This fa...
AI summary The Evaluator calculates net avoided GHG emissions for EPI by multiplying energy savings with Nova Scotia Power (NSP) data. The 2024-2025 DSM MA document guides energy savings calculations and effective useful life values. The Evaluator assesses study appropriateness based on factors like measure similarity, climate conditions, and methodology quality.
Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were condu...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence level in evaluating installation rates, free-ridership, and spillover levels. Only random sampling errors were considered, excluding non-sampling errors like data entry issues or response inaccuracies.
14.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification an...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of program results. Corrective actions are detailed in Appendix XIII, with reported savings based on adjusted data.
14.2 Gross Savings For EPI, gross savings correspond to the change in energy consumption resulting from installing energy efficient products in participant homes regardless of why they participated.[35](#page-83-2) The Evaluator relied on...
AI summary Gross savings for EPI are calculated based on energy consumption changes from efficient product installations, using the 2024-2025 DSM MA and a 2019 adjustment ratio. EUL values for lighting products were updated, with other product EUL values sourced from the same DSM MA.
up> Definition adapted from: National Renewable Energy Laboratory, The Uniform Methods Project Chapter 23: Estimating Net Savings: Common Practices, September 2014, p.3. › Foam gaskets: The installations of foam gaskets could not be visual...
AI summary The Evaluator estimated foam gasket installation rates at 29% using verbal confirmation, an uncertainty factor of 50%, and average installation data. Participants struggled to confirm installations, leading to the uncertainty adjustment.
14.3 Net Savings The Evaluator determined the net energy and peak demand savings, i.e. the electrical energy and peak demand savings that can be reliably attributed to a program component, by estimating the NTGR. In the case of EPI, the NT...
AI summary The Evaluator calculated net energy and peak demand savings by estimating the NTGR, accounting for free-ridership and participant spillover effects in EPI programs. This approach adjusts for unintended influences on energy savings outcomes.
14.3.2 Participant Spillover For EPI, participant spillover occurs when participants purchase and install additional energy efficient products due to the influence of having participated in the program component without receiving any addit...
AI summary The document discusses participant spillover in the EPI program, where participants install additional energy-efficient products post-program participation without additional support. The 2024 evaluation identified LED lamps, fixtures, and heat pump water heaters as the main products driving spillover, with 17 out of 100 surveyed participants reporting such behavior.
15.1 Jurisdictional Scan This section outlines the findings of the jurisdictional scan conducted to identify opportunities for electricianinstalled measures. In 2023, E1 introduced some electrician-installed measures to the EPI offering an...
AI summary A jurisdictional scan by E1 identified few new electrician-installed measures for EPI beyond existing offerings. Reviews of Canadian and US direct install and instant rebate programs found most measures either already included in EPI or outside electrician scope (e.g., insulation, appliances). Key findings highlight limited opportunities for expansion.
16 EPI Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 EPI evaluation were as follows: - › Collect information on participant perspectives - › Calculate gross and net EPI results, namely electrical...
AI summary The 2024 EPI evaluation aimed to collect participant perspectives, calculate energy and GHG savings, and analyze market opportunities. This section outlines key findings and recommendations related to these objectives.
2024 EPI-Finding: Participants are highly satisfied with the overall program component. During the 2024 participant survey, participants rated the overall program component a 9.4 out of 10 and also provided high ratings on the time taken t...
AI summary The 2024 EPI-Finding highlights high participant satisfaction with the program component, scoring 9.4/10. Participants praised work quality, time efficiency, and information on energy-efficient products. The finding emphasizes positive feedback on installation quality and product efficiency.
2024 EPI-Finding: Evaluated net electrical energy and peak demand savings were respectively lower and slightly higher than the values tracked by E1. The evaluated net electrical energy savings were 6% lower than the tracked net electrical...
AI summary The 2024 EPI-Finding reports evaluated net electrical energy savings were 6% lower than E1-tracked values, while peak demand savings were 1% higher. Key factors include a 16% lower smart thermostat installation rate for MSHPs and a 4% reduction in LED lamp usage hours. The NTGR value for 2024 was slightly lower than in 2023.
17 Green Heat Overview This section describes Green Heat, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Green Heat program, addresses past evaluation recommendations, and reviews participation history, highlighting its role in energy efficiency and greenhouse gas reduction initiatives in Nova Scotia.
Table 42: 2024 Green Heat Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › Which participants did not add an extensi...
AI summary This section outlines the evaluation approach for the 2024 Green Heat program, focusing on calculating gross and net results through methods such as tracking sheet audits, billing analysis, and GHG emission reduction calculations. It addresses research questions related to data accuracy, savings from mini-split heat pumps, and the evaluation of first-year and lifetime energy savings.
Past Participant Survey A total of 240 Green Heat participants composed of 204 MHSP participants and 36 wood/pellet stove or fireplace insert participants took part in a web survey in October 2024. The past participant survey served to col...
AI summary A survey of 240 Green Heat participants was conducted in October 2024 to gather insights on home renovations, including the installation of heat pumps or wood stoves, and prior air conditioning usage. The results are detailed in appendices, with Table 43 showing sample sizes and margins of error, noting that biomass measures had a sample size slightly below the 10% margin of error target.
Measure Population Included in Billing Analysis Sample Size Sampling Margin of Error at 90% Confidence Level MSHPs 1,132 204 ±5.2% Biomass 213 36 ±12.9% Unitary Savings Review
AI summary The table presents data on the sampling margin of error for two measures: MSHPs and Biomass. The data is part of a Unitary Savings Review, which evaluates the effectiveness of energy efficiency measures.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in the 2024 Green Heat evaluation but faced high variability in savings data. Results with higher margins of error were retained as statistically significant estimates, excluding the absence of savings. Confidence levels only account for random sampling errors, not nonsampling biases.
19 Green Heat Impact Evaluation The objectives of the 2024 Green Heat impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as effective useful life (EUL) value...
AI summary The 2024 Green Heat Impact Evaluation aims to assess gross and net electrical energy savings, peak demand reductions, annual GHG emission avoidance, effective useful life (EUL) values, and lifetime energy savings from the program.
19.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification an...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of program component results. Corrective actions are detailed in Appendix XXII, leading to corrected tracked savings presented in the report.
19.2.2 Unitary Energy Savings To establish Green Heat unitary savings, the Evaluator relied on a combination of billing analyses, energy models, engineering algorithms, and literature reviews. The 2024-2025 DSM MA provides a detailed descr...
AI summary The Evaluator used billing analyses, energy models, engineering algorithms, and literature reviews to establish Green Heat unitary savings. The 2024-2025 DSM MA details inputs and calculations for measure categories, with subsections focusing on MSHP measures, wood/pellet stoves, and literature-based updates to unitary savings.
Billing Analysis The billing analysis consisted of calculating the change in electrical energy consumption by comparing levels before and after participation in Green Heat for a group of recent participants (treatment group). To account fo...
AI summary The billing analysis used a difference-in-differences approach to compare energy consumption changes in Green Heat participants (treatment group) against historical participants (control group), isolating program-specific savings while accounting for external factors like the pandemic. This method updated unitary savings for MSHPs and wood/pellet heating systems, excluding participants who expanded heated areas.
20 Green Heat Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 Green Heat evaluation were as follows: › Calculate gross and net results, namely electrical first-year and lifetime energy savings, pea...
AI summary The 2024 Green Heat program achieved only 24% and 55% of its electrical energy and peak demand savings targets, respectively. Net savings were 0.872 GWh and 1.529 MW, below planned 3.600 GWh and 2.775 MW. Low participation and poor realization rates from billing analyses were primary causes.
2024 Green Heat - Finding: Green Heat participation decreased for the third consecutive year. Green Heat participation levels decreased by 21% compared to 2023 levels, particularly for MSHPs (19% reduction) and demand reduction measures (3...
AI summary Green Heat participation fell 21% in 2024, the third consecutive year of decline, attributed to federal CGH Grant competition. Evaluation showed 35% net energy savings realization, with MSHPs and biomass measures underperforming. Recommends removing wood/pellet inserts due to negligible savings.
21 HEA Overview This section describes Home Energy Assessment (HEA), follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Home Energy Assessment (HEA) program, addresses past evaluation recommendations, and summarizes participation history. It serves as an overview of the program's implementation and effectiveness.
21.1 HEA Description HEA is a home energy evaluation-based program component that encourages homeowners to improve the energy efficiency and comfort of their homes by providing them with related information and financial incentives in the...
AI summary The Home Energy Assessment (HEA) program evaluates homes for energy efficiency, offering rebates and guidance for improvements. Administered with Natural Resources Canada (NRCan), it includes pre- and post-retrofit assessments, with eligibility for rebates and the Moderate Income Rebate (MIR). The Canada Greener Homes (CGH) Grant, co-delivered via HEA, closed in 2024, removing certain measures. HEA aimed for 19.012 GWh energy savings and 4.799 MW peak demand reduction in 2024.
Table 53: 2024 HEA Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › Which participants did not add an extension as p...
AI summary Table 53 outlines the evaluation approach for the 2024 Home Energy Assessment (HEA) program, focusing on calculating gross and net results through methods like tracking sheet audits, billing analysis, and surveys. Key considerations include data accuracy, adjustment ratios, and energy savings calculations.
Expired Participant Survey To reassess unconverted D assessment savings parameters, Narrative Research conducted a telephone survey with a total of 76 expired participants, i.e. participants who did not complete a final home energy assessm...
AI summary Narrative Research conducted a telephone survey of 76 expired participants in E1's DSM program who did not complete final home energy assessments. The survey, part of the 2024-2025 DSM MA evaluation process, aimed to reassess unconverted D assessment savings parameters and inform energy savings calculations for prescriptive measures.
Calculations Using Evaluation Results Building on all the above methods and collected data, the Evaluator calculated the first-year and lifetime electrical energy and peak demand savings as per the calculation methodology presented in Sect...
AI summary The Evaluator calculated first-year and lifetime electrical energy and peak demand savings using a methodology outlined in Section 23. These calculations build on prior data collection and evaluation methods to quantify program impacts.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The evaluation of the 2024 HEA program aimed for a 10% margin of error at 90% confidence, acknowledging sampling errors but excluding nonsampling biases. Margins of error for adjustment ratios are detailed, with data sourced from Nova Scotia Power and Emera Inc.
23 HEA Impact Evaluation The objectives of the 2024 HEA impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as EUL values and associated lifetime energy savin...
AI summary The 2024 HEA impact evaluation aimed to assess gross and net electrical energy savings, peak demand reductions, annual GHG emissions avoided, and Effective Useful Life (EUL) values with associated lifetime energy savings from Home Energy Assessments.
23.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification an...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1. The audit revealed the complexity of the HEA tracking sheet, prompting the development of a new savings calculation approach. Transitioning to E1's Customer Information System (CIS) is expected to simplify data tracking, particularly for heat pump parameters, by enabling automated validation against NEEP's specifications.
23.2 Gross Savings For HEA, gross savings correspond to the change in energy consumption resulting from measures implemented by HEA participants regardless of their reasons for participating.[59](#page-156-1) The following subsections desc...
AI summary Gross savings for HEA (Home Energy Assessment) represent energy consumption changes from implemented measures, regardless of participation reasons. The methodology for calculating these savings is detailed in subsequent subsections.
Reporting Requirements HEA incentives originate from three sources of funding: Nova Scotia Power ratepayers for DSM, the Province of Nova Scotia, and the Government of Canada (CGH Grant). The incorporation of the CGH Grant, as a cofunder o...
AI summary HEA incentives are funded by Nova Scotia Power ratepayers, the Province of Nova Scotia, and the Canadian government (CGH Grant). Savings are reported to NSUARB and the Province via separate evaluations, focusing on electrical savings (DSM) and participation/GHG reductions (government). Equations prevent double-counting, and solar PV savings are included in DSM reports regardless of heating source.
23.2.4 Supplemental File Adjustments Supplemental files refer to participants that have multiple files (or lines) in the tracking sheet. Supplemental files occur for multiple reasons that may or may not result in incremental energy savings...
AI summary Supplemental files in energy efficiency programs may arise from administrative issues or home remodeling, impacting energy savings. E1 analyzed 2024 adjustments, identifying incremental savings from updated files. The Evaluator deems E1's approach thorough, yielding positive adjustments to gross savings, as shown in Table 58.
23.3.2 Participant Spillover For HEA, participant spillover occurs when participants implement additional measures recommended in their initial energy assessments after their participation in the program component, that is after having com...
AI summary Participant spillover in the Home Energy Assessment (HEA) program occurs when participants implement additional energy efficiency measures after completing their initial assessments, without receiving further support from the program. The 2023 spillover level was reused in the 2024 evaluation due to a lack of updated data collection.
24 HEA Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 HEA evaluation were as follows: › Calculate gross and net results, namely electrical first-year and lifetime energy savings, peak demand savin...
AI summary The 2024 HEA evaluation found that net electrical energy and demand savings exceeded targets, with participation reaching a historic high due to the CGH Grant. Solar PV measures contributed significantly to savings, but future participation may decline post-CGH Grant closure. Energy savings tracked by E1 were lower than evaluation results, prompting adjustments. A recommendation to remove wood/pellet fireplace inserts from HEA is proposed due to low savings.
2024 HEA-Finding: Parameters used to calculate unconverted D assessment spillover were reassessed and remain relatively stable. As part of the 2024 evaluation, the Evaluator compiled the results of 19 site visits completed by EAs among sur...
AI summary The 2024 HEA evaluation reassessed parameters for unconverted D assessment spillover using 19 site visits by Energy Auditors (EAs) on expired participants. While falling short of the 30-visit target, results were deemed consistent and sufficient. E1 may seek additional visits in 2025 for higher precision.
25 MHEEP Overview This section describes the Mi'kmaw Home Energy Efficiency Project (MHEEP or the Project) program component, follows up on past evaluation recommendations, and provides an overview of MHEEP participation history.
AI summary This section outlines the Mi'kmaw Home Energy Efficiency Project (MHEEP), detailing its program component, follow-up on prior evaluation recommendations, and an overview of historical participation in the initiative.
25.2 Follow-up on Past Evaluation Report Recommendations No recommendations were made for MHEEP in the 2023 evaluation.
AI summary The 2023 evaluation of the Mi'kmaw Home Energy Efficiency Project (MHEEP) did not result in any recommendations being made, indicating that the program's performance or outcomes were either satisfactory or not requiring further action based on the assessment.
25.3 Participation History As presented in [Figure](#page-42-0) 16 below, 192 participants completed projects and achieved electrical energy savings under MHEEP in 2024, representing a 19% increase in participation compared to 2023 and the...
AI summary The MHEEP program saw a 19% increase in participants in 2024, but average energy savings per participant dropped by 41% due to updated adjustment ratios. Appliance replacements also occurred, and peak demand savings increased despite lower energy savings.
Table 70: 2024 MHEEP Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › What are the savings for heating and building...
AI summary The document outlines the evaluation approach for the 2024 MHEEP (Mi'kmaw Home Energy Efficiency Project), focusing on calculating gross and net results through data audits, unitary savings reviews, effective useful life updates, and GHG emission reduction calculations.
27 MHEEP Impact Evaluation The objectives of the 2024 MHEEP condensed impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as EUL values and associated lifetim...
AI summary The 2024 MHEEP condensed impact evaluation assesses electrical energy and peak demand savings, annual GHG emission reductions, and EUL values. The report focuses on standard-allocation electrical savings, excluding nonelectrical energy savings from installed measures.
27.1 Tracking Sheet Audit To ensure project results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The results obtained from the...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable project results. Corrected tracked savings are detailed in Appendix XXXIV, forming the basis of the report's findings.
27.2 Gross Savings MHEEP gross savings correspond to the change in energy consumption resulting from the measures implemented by participants regardless of why they participated.[64](#page-177-0) In 2024, MHEEP participants received buildi...
AI summary MHEEP gross savings are calculated based on energy consumption changes from implemented measures, including building envelope and heating upgrades. Non-modeled measures use unitary savings values. Methodologies are detailed in subsections, referencing NREL's Uniform Methods Project for net savings estimation.
Where: - › and correspond to the modelled energy consumption levels respectively obtained in HOT2000 during the pre-retrofit (D) and post-retrofit (E) assessments. When modelled energy consumption levels were not available, the participant...
AI summary The text outlines methods for calculating energy savings using HOT2000 and EnerGuide ratings, unitary savings values for HPWHs and DWHRs, adjustment ratios (ARs) derived from billing analyses, and prescriptive measure calculations from MHEEP. ARs vary by heating system scenarios and are based on 2024 HEA evaluations.
27.2.3 Interactive Effects In a home, interactive effects occur when the implementation of energy efficiency measures has an impact on the energy consumption of other elements such as heating and cooling. The interactive effects of the spa...
AI summary Interactive effects in energy efficiency measures, such as those modeled in HOT2000, are considered in savings calculations. However, for the 2024 evaluation, certain measures like programmable thermostats and drain water heat recovery systems, as per the 2024-2025 Measure Assessment, do not impact other elements' energy consumption.
27.3.1 Evaluated Net Savings Net savings are defined as the energy savings specifically attributable to MHEEP. Since spillover and freeridership effects were considered nil, the net MHEEP impacts are equal to the gross savings generated by...
AI summary The MHEEP program achieved net energy savings of 0.401 GWh and peak demand savings of 0.481 MW, with no spillover or free-rider effects. It missed its energy target by 35% but exceeded peak demand savings by 189% as shown in Figure 55.
28 MHEEP Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 MHEEP evaluation were as follows: › Calculate gross and net MHEEP results, namely electrical first-year and lifetime energy savings, peak de...
AI summary The 2024 MHEEP evaluation found that net electrical energy savings fell short of targets (65% achieved vs. 95% in 2023), while peak demand savings exceeded targets. Higher participation (19% increase) was offset by a 41% drop in average savings per participant due to updated adjustment ratios. Evaluated savings were 31% lower and 15% higher than E1's initial tracking for energy and peak demand, respectively.
29 Residential Behaviour Overview This section describes the Residential Behaviour program component, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Residential Behaviour program, its follow-up on past evaluations, and provides an overview of participation history within the program.
29.1 Description The Residential Behaviour program component, publicly branded as Efficiency Insights, is designed to help Nova Scotia Power (NS Power) residential customers reduce their electricity consumption. The component provides a su...
AI summary The Residential Behaviour program, branded as Efficiency Insights, helps NS Power customers reduce electricity use via personalized Home Energy Reports (HERs) and behavior tips. E1, using Bidgely's algorithms, integrates HERs into NS Power's MyEnergy Insights platform. Energy Solutions Advisors (ESAs) assist customers, and savings are evaluated annually through billing analysis. Funded under E1's 2023-2025 DSM Plan, the program launched in May 2024 with four reports issued in 2024.
29.2 Follow-up on Past Evaluation Report Recommendations There are no past recommendations since this is the first evaluation of Residential Behaviour.
AI summary The section states there are no past recommendations to follow up on, as this is the first evaluation of Residential Behaviour initiatives.
Analysis of Participation in Other Programs The Evaluator compared the participation levels between the treatment and control groups in other residential program components offered by E1, namely Efficient Product Installation (EPI), Green...
AI summary The Evaluator analyzed participation rates in E1's residential programs (EPI, Green Heat, HEA) between treatment and control groups to verify higher engagement in treatment groups and adjust savings calculations to prevent double-counting.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The evaluation of the 2024 Residential Behaviour program reports a 10% margin of error at 90% confidence, accounting only for random sampling errors. Nonsampling errors (e.g., data entry, response bias) are excluded. Margins of error accompany savings values from billing analysis, emphasizing precision over accuracy.
31 Residential Behaviour Impact Evaluation The objectives of the 2024 Residential Behaviour impact evaluation were to determine net electrical energy savings. The savings calculation methodology for Residential Behaviour is based on evalua...
AI summary The 2024 Residential Behaviour Impact Evaluation aimed to quantify net electrical energy savings using a randomized controlled trial (RCT) methodology, aligning with industry best practices and evaluation protocols to ensure unbiased savings estimates through comparison of treatment and control groups.
31.1 Tracking Sheet Audit Considering Residential Behaviour relies on a random selection of treatment group participants among all residential customers, the program component does not have a tracking sheet. Therefore, no tracking sheet au...
AI summary The Residential Behaviour program component does not require a tracking sheet audit because it uses random selection of participants from all residential customers, eliminating the need for targeted tracking sheets.
31.2 Net Savings For Residential Behaviour, savings correspond to the change in electricity consumption resulting from behaviours adopted by treatment group participants compared with the change in electricity consumption observed among co...
AI summary Net savings for residential behavior programs are calculated by comparing electricity consumption changes between treatment and control groups, ensuring savings directly attribute to the program. The Uniform Methods Project (UMP) defines net savings as the difference in energy consumption with and without the program, excluding control group changes. No additional freeridership adjustments are needed, but overlap with other ENS programs must be considered.
31.2.3 Energy Savings The model used to calculate savings for Residential Behaviour is the difference-in-difference (DiD). The DiD serves to compare the average change in electricity consumption in the treatment group prior to and during p...
AI summary The Residential Behaviour program uses a difference-in-difference (DiD) model to calculate energy savings, comparing treatment and control groups across preprogram (May 2023–April 2024) and post-program (May–December 2024) periods. Monthly savings trends are analyzed, but cumulative savings are reported officially due to statistical insignificance in some monthly results.
Data Preparation Before calculating savings, the Evaluator cleaned and prepared the AMI data provided by E1. The received AMI data contained consumption data aggregated on a monthly basis. The pre-program data cover the 12-month period pri...
AI summary The Evaluator cleaned AMI data from E1, removing outliers, duplicates, and inactive accounts, while retaining opted-out customers to avoid bias. Pre-program data spanned May 2023–April 2024, and post-program data covered May–December 2024. Attrition rates were 5.6% for customers and 0.6% for observations.
Cumulative Savings The DiD model serves to compare the difference in the average daily consumption between the treatment group and the control group before and after program participation. The cumulative approach uses the average daily con...
AI summary The document explains the use of a Difference-in-difference (DiD) model to evaluate cumulative savings by comparing average daily consumption between treatment and control groups. It also outlines the cumulative approach equation for calculating total savings across the entire period, with monthly savings details in Appendix XXXVI.
31.2.6 Effective Useful Life For Residential Behaviour, energy savings are assessed annually through a billing analysis that serves to calculate the change in electricity consumption between the program evaluation year (post-program period...
AI summary The text explains that residential behavior programs in Nova Scotia use an Effective Useful Life (EUL) of one year for energy savings calculations, based on annual billing analysis comparing pre- and post-program periods. This approach aligns with practices in other jurisdictions like Massachusetts and Illinois.
31.2.7 Savings Deductions for Participation in Other Residential Programs As mentioned in Subsection [29.1](#page-186-1) above, a secondary aim of Residential Behaviour is to encourage customers to engage with other ENS programs tailored t...
AI summary The Residential Behaviour program aims to encourage participation in other ENS programs. Savings deductions are calculated if treatment groups show higher participation in EPI and Green Heat programs. Statistically significant differences were found in participation levels for high and medium users in Green Heat and EPI, requiring savings deductions for those waves.
2024 Residential Behaviour-Finding: Residential Behaviour fell short of its net electrical energy savings targets. Residential Behaviour achieved 6.270 GWh in net electrical energy savings in 2024, thus falling short of the planned net ele...
AI summary Residential Behaviour program underperformed in 2024, achieving 6.270 GWh vs target 8.000 GWh, but shows potential as it scales. Control and treatment groups were similar, and treatment group had higher participation in other programs.
Affordable Single-family Homes Appendix VII: ASFH Participant Telephone Survey Questionnaire Appendix VIII: ASFH Participant Survey Results Appendix IX: ASFH Heat Pump Contractor Interview Guide Appendix X: ASFH Delivery Agent Interview Gu...
AI summary The document outlines appendices related to the Affordable Single-family Homes (ASFH) program, including survey questionnaires, interview guides, audit tracking sheets, and 2024 recommendations, supporting data collection and evaluation efforts for the initiative.
Efficient Product Installation Appendix XIII: EPI Tracking Sheet Audit Appendix XIV: EPI Participant Survey Questionnaire Appendix XV: EPI Participant Survey Results Appendix XVI: EPI On-site Visit Sampling Methodology and Protocol Appendi...
AI summary The document outlines appendices related to Efficient Product Installation (EPI), including audit processes, participant surveys, free-ridership/spillover calculation algorithms, jurisdictional findings, and 2024 recommendations. These materials support program evaluation and implementation methodologies.
Green Heat Appendix XXI: Green Heat Past Participant Survey Questionnaire Appendix XXII: Green Heat Past Participant Survey Results Appendix XXIII: Green Heat Tracking Sheet Audit Appendix XXIV: Green Heat Detailed Methodology and Billing...
AI summary The document outlines appendices related to the 'Green Heat' initiative, including past participant surveys, tracking sheets, methodology, billing analysis, and 2024 recommendations, indicating a focus on program evaluation, data collection, and implementation strategies.
Home Energy Assessment Appendix XXVI: HEA Past Participant Survey Questionnaire Appendix XXVII: HEA Past Participant Survey Results Appendix XXVIII: HEA Expired Participant Survey Questionnaire Appendix XXIX: HEA Expired Participant Survey...
AI summary The document outlines appendices related to the Home Energy Assessment (HEA) program, including past and expired participant surveys, audit tracking sheets, billing analysis methodologies, reporting requirements, and 2024 recommendations. These appendices focus on program evaluation, performance monitoring, and participant engagement metrics.
Mi'kmaw Home Energy Efficiency Project Appendix XXXIV: MHEEP Tracking Sheet Audit Appendix XXXV: MHEEP 2024 Recommendations
AI summary The document includes appendices related to the Mi'kmaw Home Energy Efficiency Project (MHEEP), focusing on tracking sheets and 2024 recommendations, indicating regulatory oversight and evaluation of the program's implementation and outcomes.
2024 DSM EVALUATION March 25, 2025 In Collaboration with:
AI summary The 2024 Demand-Side Management (DSM) evaluation by Nova Scotia Power outlines program performance and outcomes, with collaboration noted in the document. Key focus areas include energy efficiency initiatives, program effectiveness, and regulatory compliance.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to verify the accuracy and completeness of data submitted by EfficiencyOne (E1) for program evaluation, ensuring consistency in calculation methods and parameters used for assessing energy savings.
Introduction Please could I speak with ? - 1. Yes [CONTINUE] - 2. Not available [ASK WHEN PERSON MIGHT BE AVAILABLE] Hello, my name is ____________________ and I'm calling from Narrative Research, a Halifax based market research company, o...
AI summary The text is a script for contacting program partners to evaluate Efficiency Nova Scotia's Affordable Multifamily Housing Program, which offers free energy audits and rebates for energy upgrades. The goal is to gather opinions through 15-minute interviews.
D. Challenges/Barriers to Completion - D1. In your experience, what reasons would lead someone who completed an energy audit of their building to choose not to go ahead with the recommended upgrades and discontinue their participation in t...
AI summary The section outlines open-ended questions about barriers to program completion, including reasons participants might discontinue upgrades post-audit, challenges faced, requested support (e.g., better communication), and strategies for Efficiency Nova Scotia to improve participation rates.
E. Satisfaction - E1. On a scale of 0 to 10, where 0 is "Not at all satisfied" and 10 is "Completely satisfied," how satisfied are you with each of the following ten aspects of the Affordable Multifamily Housing program? [DO NOT RANDOMIZE]...
AI summary The text outlines a satisfaction survey for the Affordable Multifamily Housing program, evaluating aspects like information provision, audit processes, software tools, and communication with Efficiency Nova Scotia, including open-ended feedback opportunities.
F. Recommendations for Program Improvements - F1. What information, tools, support, or training would make your role in the Affordable Multifamily Housing program easier? [OPEN END] - F2. Do you have any ideas how to better communicate or...
AI summary The section seeks stakeholder input on improving the Affordable Multifamily Housing (AMH) program through better tools, communication strategies, and additional recommendations. Four open-ended questions focus on enhancing program effectiveness, participant engagement, and overall improvements.
[0-10 SCALE WITH END POINT LABELS– RECORD NUMBER] - 98. Don't Know - 99. Refused - C3. [IF [D<](#page-13-1)8] What is the main reason you are not more satisfied with the program overall? Are there other reasons? [PROBE FOR SPECIFIC REASON....
AI summary The text outlines a survey assessing customer satisfaction with Nova Scotia's Affordable Multifamily Housing program, focusing on aspects like application processes, rebate satisfaction, and form usability. It includes 0-10 scales, open-ended responses, and specific probes for dissatisfaction reasons.
B. Reasons for Ending / Barriers to Participation - B1. [ASK DROPPED OUT PARTICIPANTS ONLY] The information I have shows that you have not continued your participation in the Affordable Multifamily Housing program for the building at < INS...
AI summary The document outlines reasons for participants dropping out of the Affordable Multifamily Housing (AMH) program, including communication issues, ineligibility, application difficulties, and dissatisfaction with program terms. It seeks detailed feedback on the stage of exit and specific barriers encountered.
[Problems pre-renovation assessment] - 19. It took too long to schedule the initial pre-renovation assessment - 20. The advisor I dealt with during the pre-renovation assessment process was not knowledgeable - 21. The advisor I dealt with...
AI summary Customers reported significant issues with the pre-renovation assessment process, including long scheduling delays, unprofessional and uninformed advisors, poor communication, lack of punctuality, incomplete assessments, and disrespectful behavior toward properties.
C. Benefits and challenges The following series of questions will be focused on the benefits you received and challenges you may have experienced from participating in the Home Warming program. - C1. On a scale of 0 to 10, where 0 is "do n...
AI summary The text outlines a survey methodology to assess participant experiences with the Home Warming program, focusing on perceived benefits (comfort, cost savings, space utilization, winter warmth) and challenges. It includes structured questions with scales and open-ended responses to gather qualitative and quantitative feedback.
D. Opportunities for Improvement The following series of questions will be focused on improving the HomeWarming program. - D1. [ASK IF < MEASURE CATEGORY = MODELLED > ] What questions, if any, do you have about the different upgrades insta...
AI summary The text outlines opportunities to improve the HomeWarming program by addressing participant questions about upgrades, heat pump installations, and communication preferences. Recommendations include expanding available upgrades, simplifying processes, improving product information, and enhancing stakeholder interaction.
C. Program Strengths and Challenges Now, I would like to discuss the HomeWarming program – heat pump delivery process with you. In general, we understand that a project includes the following steps: [READ]The contractor receives informatio...
AI summary The HomeWarming program's heat pump delivery process involves contractor coordination with Efficiency Nova Scotia (ENS) for participant outreach, sizing, installation, and reimbursement. The text seeks confirmation of the process and feedback on its effectiveness, challenges, and potential improvements from contractors and participants.
Aspects of the Program Score 0 = Not at all satisfied 10 = Completely Satisfied Reason If 7 or less, please share the reason(s) for your score. a. The HomeWarming program overall /10 What could be improved? b. HW participation levels /10 W...
AI summary The HomeWarming program and related aspects such as participation levels, range of upgrades, product quality, and communication effectiveness are being evaluated. Stakeholders are asked to identify areas for improvement.
This appendix summarizes all the recommendations made by the Evaluator as part of the 2024 ASFH evaluation. Section Recommendations Executive Summary Recommendation #1: To assess progress with participant wait times, E1 could track wait ti...
AI summary The appendix outlines seven recommendations from the Evaluator for improving the 2024 ASFH evaluation, focusing on participant experience, program clarity, communication, and administrative efficiency for delivery agents and contractors.
C. Satisfaction C1. Using a scale from 1 to 10 where 1 is "not at all satisfied" and 10 is "completely satisfied," how satisfied are you with the Efficient Product Installation Service overall? [DO NOT ACCEPT A RANGE] 1-10 SCALE WITH END P...
AI summary The section evaluates customer satisfaction with the Efficient Product Installation Service (EPI) through 1-10 and 1-4 scales, exploring reasons for satisfaction or dissatisfaction, focusing on energy bill savings, service quality, and product effectiveness.
[ASK [E4,](#page-106-1) [E5,](#page-107-0) [E6](#page-107-1) SEQUENCE IN ORDER/DO NOT RANDOMIZE; REPEAT SCALE IF NECESSARY] - E4. Without the Efficient Product Installation Service, how likely would you have been to take the initiative to...
AI summary The text presents survey questions assessing the impact of the Efficient Product Installation Service (EPI) on consumer behavior regarding LED bulb adoption. Respondents are asked about likelihood of purchasing, delaying replacement, and quantity purchased without EPI, evaluating its role in promoting energy efficiency.
[VOLUNTEERED] - 98. Don't know - 99. Refused - E14. [IF [E13=](#page-109-1)YES] Just to confirm: Have I understood correctly that you had already made the decision to purchase and install in your home before you learned about the Efficient...
AI summary The text contains survey questions assessing customer decision-making regarding home energy efficiency measures, including willingness to pay for specific installations (e.g., pipe insulation, hot water tank wrap) with and without the Efficient Product Installation Service (EPI). It explores pre-existing decisions and hypothetical scenarios without EPI incentives.
Sampling Methodology For the 2024 EPI impact evaluation, 60 on-site visits were conducted with participants. The selected samples included 60 EPI participants as well as 60 back-up participants that had at least four different products ins...
AI summary The 2024 EPI impact evaluation involved 60 on-site visits with 60 EPI participants and 60 backup participants, ensuring a representative sample with at least four product installations each. The sample was verified to be within ±10% of the proportion served by delivery agents and included a diverse mix of product installations.
On-site Visit Protocol The on-site visits were conducted to verify the number of products installed and recorded in the tracking system. The on-site observation protocol was developed to serve two main purposes: (1) Establish the installat...
AI summary The on-site visit protocol outlines procedures to verify installed product quantities, assess installation rates, and identify product removals. Evaluators collect data on lamp locations, domestic hot water systems, faucet configurations, and heating sources to validate assumptions about energy savings calculations.
APPENDIX XVIII EPI Algorithm for Participant Spillover Calculation Participant spillover was measured using a participant survey. Participants were asked, pursuant to participating in EPI, whether they implemented any additional energy eff...
AI summary This appendix describes the EPI Algorithm for calculating participant spillover, which measures the impact of energy efficiency programs on participants' additional energy efficiency measures. Surveys were used to determine the influence of program components on participants' decisions, and spillover was calculated by dividing additional savings by total savings achieved through program participation.
A. Verification and Recall - A1. We understand you participated in the program in for your house located at [IF GH, ADD] by installing a . Is that correct? - 1. Yes [CONTINUE] - 2. No [END]
AI summary This section outlines a verification process for program participation, asking respondents to confirm installation of specific energy efficiency measures in their homes as part of a program during a specified year. The text includes a yes/no response option with branching logic.
APPENDIX XXIV Green Heat Detailed Methodology and Billing Analysis
AI summary Appendix XXIV outlines the methodology and billing analysis for the Green Heat initiative, focusing on energy efficiency programs, appliance retirement, and heat pump adoption in Nova Scotia. It includes technical details on cost recovery, program evaluation, and regulatory considerations for residential and commercial energy efficiency measures.
Methodology The billing analysis consisted of calculating the change in electrical energy consumption by comparing levels before and after participation in Green Heat for a group of recent participants (treatment group). To account for var...
AI summary The methodology uses a difference-in-differences approach to measure Green Heat program savings by comparing treatment and control groups, normalizing energy data for weather, and calculating unitary savings for MSHPs and wood/pellet stoves. This isolates program impacts from external factors like the pandemic.
Selection of the Treatment and Control Groups The initial treatment group included 1,211 Green Heat participants (1,014 for MSHP measures and 197 for wood/pellet stoves/fireplace inserts) who were selected based on the following criteria:...
AI summary The treatment group included 1,211 Green Heat participants with specific installation dates and heating criteria, while the control group had 2,810 past participants selected to match treatment characteristics. AMI data availability varied between groups, and the Evaluator ensured similarity using UMP guidelines.
Control-Treatment Participant Matching To determine the best suited control group participant for each treatment group participant, the Evaluator first applied a set of minimum conditions with which to comply: - › The installation date of...
AI summary To match control and treatment participants, the Evaluator applied specific conditions, including installation date gaps, identical weather stations, and matching measures. A point system was used to score potential control participants, with the highest-scoring match assigned to each treatment participant.
Energy Savings Calculation Figure 1 below illustrates how electrical energy savings were obtained by calculating the difference in the treatment group normalized annual consumption of each participant for the year before and after program...
AI summary The text describes energy savings calculations using a difference-in-differences approach with AMI data, normalizing consumption before/after program participation and adjusting for control group trends. Unitary savings values are calculated per MSHP capacity and per unit for wood/pellet stoves.
Use of Past Participant Survey Results Given the inconclusive results of 2023 billing analysis, it was hypothesized that some participants included in the billing analysis may have added heated floor area to their home around the same time...
AI summary The 2023 billing analysis for Green Heat had inconclusive results, possibly due to some participants adding heated floor area post-participation, increasing consumption. A survey was conducted to exclude such cases, but limited results made it incomplete.
Summary of Literature Review Findings The Evaluator reviewed the aforementioned three studies to compare and put into perspective the savings results of the 2024 Green Heat billing analysis. - › The 2024 billing analysis conducted in Orego...
AI summary The Evaluator compared the 2024 Green Heat billing analysis with studies from Oregon, Massachusetts/Connecticut, and Vermont. Results showed similar savings (1,032–1,075 kWh) except for Vermont’s higher savings (3x), attributed to different methodologies, higher-efficiency heat pumps, and early adopter behavior. Recent studies reinforce confidence in Green Heat’s findings.
C. Reasons for Not Conducting an E Assessment - C1. Which one of the following best describes the reason why you chose not to have the follow-up visit by an Energy Advisor? [DO NOT READ, ACCEPT ONE RESPONSE] - 1. Follow-up visit was too di...
AI summary The document presents survey questions to identify reasons participants did not pursue follow-up energy assessments, including scheduling difficulties, cost concerns, perceived lack of utility, and completion of energy-saving measures. Options highlight barriers to program participation and evaluation.
› For the differences in gross energy savings: - › Correction of unitary savings for some biomass measures - › Correction of the logic used to determine the unitary savings for HPWH (heat pump or electric baseboard baseline heating system)
AI summary The text outlines two corrections related to energy savings calculations: adjusting unitary savings for biomass measures and revising the logic for determining unitary savings for HPWH (Heat Pump Water Heater) under different baseline heating systems.
› For the differences in net energy savings: - › Correction of NTGR and/or line loss factors for some participants that had hardcoded values using outdated parameters instead of formulas - › A deduction of additional participants for uncon...
AI summary The text outlines corrections needed for net energy savings calculations: adjusting NTGR/line loss factors using outdated parameters, addressing unconverted D assessment spillover reversals, and fixing Green Heat savings deductions that incorrectly used gross meter savings instead of net generator savings in EfficiencyOne's tracking sheet.
APPENDIX XXXI HEA Detailed Billing Analysis Methodology and Results
AI summary This appendix outlines the methodology and results of a detailed billing analysis for Home Energy Assessments (HEA), evaluating energy efficiency program impacts through data analysis techniques and outcome measurements.
Background After inconclusive results in 2023, a billing analysis was again conducted for HEA as part of the 2024 DSM evaluation to obtain measured electrical energy savings generated through the installation of energy efficiency upgrades....
AI summary The 2024 DSM evaluation shifted from using overestimation ratios (ORs) based on space heating data to adjustment ratios (ARs) derived from AMI data for whole-home consumption. This change addresses challenges posed by increased heat pump adoption and simplifies calculations by directly applying ARs to modelled savings estimates.
Selection of the Treatment and Control Groups The initial treatment group included 1,803 HEA participants who were selected based on the following criteria: - › Be 100% electrically heated according to HOT2000 (meaning that the primary hea...
AI summary The treatment group of 1,803 HEA participants was selected based on heating type, assessment dates, and AMI data availability. The control group of 1,227 past HEA participants was chosen to match the treatment group, ensuring similar characteristics per UMP guidelines. AMI data was incomplete due to missing NS Power account numbers, affecting the number of participants analyzed.
AMI Data Preparation AMI data for treatment group and control group participants were obtained from two sources: - › NS Power for the period between the moment AMI data became available (2020) and November 2023 - › EfficiencyOne for the ye...
AI summary AMI data for treatment and control groups were sourced from NS Power (2020–2023) and EfficiencyOne (2024). Data was aggregated monthly, with the Evaluator excluding 5% of samples due to inconsistencies like nil consumption, extreme peaks, or abnormal variations to ensure analytical accuracy.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to verify data completeness and accuracy in the evaluation submitted by E1. The audit ensured consistency in parameters used for calculating program results and validated calculation steps.
Where: - › = A number between 1 and 12 to identify the given month - › _ℎ, = The control group's average daily consumption in a given month of the post-program period - › _ℎ, = The treatment group's average daily consumption in a given mon...
AI summary The document outlines the methodology for calculating monthly electricity savings in 2024 for high, medium, and low users, noting discrepancies in active treatment participants due to inactivity or solar transitions between group selection and program launch.
DEFINITIONS Accuracy Reflects the proximity of measurements to the true value. Evaluated savings Gross and net energy or peak demand savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, inte...
AI summary The document defines key terms related to energy efficiency program evaluation, including accuracy, evaluated savings, free-ridership, and gross savings. These definitions emphasize the importance of measuring and analyzing program impacts accurately.
Table 1: Summary of 2024 Efficient Product Rebates Program Evaluation Program Component Evaluation Type Impact Process Market Methodology BER Comprehensive (Instant Rebates) Condensed (Application Rebates) X X › Instant Rebates participant...
AI summary This table summarizes the evaluation of the 2024 Efficient Product Rebates Program, focusing on the Business Energy Rebates (BER) component. It outlines the evaluation types, including impact, process, and market, and details the methodologies used, such as participant surveys, distributor interviews, tracking sheet audits, and GHG emission reduction calculations.
1 BER Overview This section describes Business Energy Rebates (BER) program component, follows up on past evaluation recommendations, and presents participation history.
AI summary This section outlines the Business Energy Rebates (BER) program, addresses past evaluation recommendations, and provides an overview of participation history within the program.
Table 5: 2024 BER Evaluation Approach Evaluation Objectives Research Questions Methodology Collect information on participant and distributor perspectives › How do participants become aware of BER-IR? › What is the level of satisfaction wi...
AI summary Table 5 outlines the 2024 BER Evaluation Approach, focusing on collecting participant and distributor perspectives, calculating gross and net results, and using methods such as surveys, interviews, and adjustment ratios. It also references the 2024-2025 DSM MA for evaluating energy savings and GHG emission reductions.
Calculations Using Evaluation Results Building on all the above methods and collected data, the Evaluator calculated first-year and lifetime energy savings as well as peak demand savings as per the calculation methodologies presented in Se...
AI summary The Evaluator calculated first-year and lifetime energy savings, as well as peak demand savings, using methodologies outlined in Sections 4 and 5 of the document. These calculations build on prior data collection and evaluation methods.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence for quantitative evaluations. BER-AR evaluations did not require margin calculations, while BER-IR included free-ridership margin calculations detailed in Appendix II of the 2024 DSM Programs Evaluation Executive Summary.
3 Process Evaluation for Instant Rebates This section summarizes the findings obtained during the Instant Rebates participant survey and interviews conducted with Instant Rebates distributors.
AI summary This section summarizes findings from the Instant Rebates participant survey and interviews with distributors, evaluating the program's implementation and effectiveness through stakeholder feedback.
3.2 Participant Satisfaction
AI summary The section titled '3.2 Participant Satisfaction' outlines the evaluation of participant satisfaction within the regulatory proceeding, though no detailed content or findings are provided in the text.
2024 BER Participant and Distributor Perspective Highlights - › The majority of Instant Rebate survey participants purchased energy efficient products for their own organization, while fewer purchased the products for a customer/project ou...
AI summary Participants in the 2024 BER Instant Rebate program primarily purchased energy-efficient products for their own organizations, driven by energy efficiency and cost savings. Satisfaction with the service was high (9.1), though distributors suggested expanding eligibility and improving rebate processes. Distributors used E1 rebates to promote LEDs but faced challenges with administrative burdens and product adoption perceptions.
4 Impact Evaluation for Application Rebates The objectives of the 2024 Application Rebates impact evaluation were to determine gross and net electrical energy and peak demand savings.
AI summary The 2024 impact evaluation for Application Rebates aimed to assess gross and net electrical energy savings and peak demand reductions, focusing on quantifying the program's effectiveness in achieving energy efficiency goals.
4.1 Tracking Sheet Audit To ensure service results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The results obtained from the...
AI summary A tracking sheet audit was conducted by the Evaluator to verify the completeness and consistency of data submitted by E1. The audit results, presented in Appendix IV, led to corrected tracked savings figures being used in the report.
4.2.1 Adjustment Ratios The Evaluator applied adjustment ratios to the tracked gross savings to determine evaluated gross savings. These ratios were established using 2022 project review results, with the exception of the lighting measure...
AI summary Adjustment ratios were applied to tracked gross savings using 2022 project reviews, except for lighting measures using 2021 data. Solar PV projects had specific ratios (0.942 for RETScreen, 1.101 for PVWatts) not used by E1 in 2024, with a recommendation to adopt them for accuracy.
4.2.2 Interactive Effects Interactive effects occur when the implementation of energy efficiency measures has an impact on the energy consumption of other elements such as heating and cooling; these are considered in the gross savings stan...
AI summary Interactive effects in energy efficiency programs, particularly for indoor lighting under BER, are evaluated using the BER CIRx Screening Tool. Adjustments are made based on site observations and 2024 DSM MA factors, with evaluators ensuring accuracy. Lighting adjustments are reflected in the 2024 adjustment ratio.
Table 13: 2024 Application Rebates NTGR Measure Category Free-ridership Spillover NTGR All 26% 0% 0.74 4.3.4 Evaluated Net Savings
AI summary Table 13 provides information on the 2024 Application Rebates NTGR, including free-ridership and spillover percentages. Section 4.3.4 discusses the evaluated net savings, highlighting the need for consideration of these factors in rebate programs.
5.1 Tracking Sheet Audit To ensure program service results were reliably compiled, the Evaluator first performed a tracking sheet audit to verify the completeness and consistency of the data submitted by E1. The results obtained from the t...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable program service results. Corrected tracked savings, as presented in Appendix V, form the basis of the report's findings.
7.1.2 BER-IR LED Purchasing Decisions The Evaluator conducted a participant survey, including both contractors and end users, to understand the LED purchasing decisions of participants. Most participants (78%) reported that they had purcha...
AI summary The Evaluator surveyed participants (contractors and end users) on LED purchasing decisions under BER-IR. 78% replaced existing fixtures (34% end-of-life, 22% broken), 20% used for new construction. 86% would have needed new lamps without the service, highlighting program impact on replacement decisions.
7.2 Evolution of LED Versus Non-LED Lighting Products To understand the availability of LED versus non-LED products in Nova Scotia, the Evaluator asked BER-IR distributors what proportion of their 2024 lighting stock was LED versus non-LED...
AI summary The Evaluator surveyed BER-IR distributors in Nova Scotia, finding that 96% of their 2024 lighting stock was LED, with 100% for linear and high-bay fixtures. Distributors expect LED dominance to grow further by 2026, up from 92% in 2021. This reflects rapid market adoption of LED technology over non-LED alternatives.
in Instant Rebates savings generated by LED linear lamps and LED linear fixtures. 2024 BER Finding: Overall satisfaction with BER Instant Rebates is high among both participants and distributors. For Instant Rebates, average satisfaction r...
AI summary The 2024 BER evaluation highlights high satisfaction (9.1/10 by participants, 8.3/10 by distributors) with Instant Rebates, an updated free-ridership algorithm showing 8-15% free-ridership, and 4-6% higher savings in Application Rebates compared to Instant Rebates. The algorithm now balances intention and influence scores, contributing to lower free-ridership due to increased 2023-2024 incentives.
Business Energy Rebates Appendix I BER: Instant Rebates Participant Survey Questionnaire Appendix II BER: Instant Rebates Participant Survey Results Appendix III BER: Instant Rebates Interview Guide with Distributors Appendix IV BER: Appli...
AI summary The document outlines appendices for Nova Scotia's Business Energy Rebates (BER) program, including surveys, audit tracking sheets, free-ridership algorithms, and 2024 recommendations. EfficiencyOne is identified as the organization involved in the program's implementation.
Previous BER and/or ENS Program Participation Led to Assess Cost-Effectiveness of Different Energy-Efficient Options 2018 2019 2020 2021 2022 2024 Sample Size 29 37 35 29 34 28 Agree 86% 89% 94% 86% 88% 93% Disagree 7% 11% 6% 14% 12% 7% Do...
AI summary The table shows the percentage of respondents who agreed, disagreed, or were unsure about the cost-effectiveness of energy-efficient options based on their participation in the BER or ENS programs from 2018 to 2024. Agreement levels are generally high, with the highest at 94% in 2020.
A. INTRODUCTION A – Business with a contact name Could I speak with ? - 1. Yes [CONTINUE] - 2. No [SAY "PERHAPS YOU CAN HELP ME ANYWAY." GO TO INTRODUCTION B] Hello, I am with Narrative Research, and we are conducting an evaluation of ener...
AI summary This document outlines a survey conducted by Narrative Research to evaluate Efficiency Nova Scotia's Business Energy Rebates Program. It seeks input from participants who purchased rebated products, focusing on their experience with the program and its effectiveness.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to verify data completeness and accuracy of tracked results, including energy and peak demand savings, and presents corrected tracked savings in Table 1 and Table 2.
APPENDIX VI BER Instant Rebates Algorithm for Free-Ridership Calculations Table 1 and Table 2 below present the algorithm[s](#page-142-1) 1 used to calculate the free-ridership levels for Instant Rebates measures. The algorithms are based...
AI summary The appendix outlines algorithms for calculating free-ridership levels in BER Instant Rebates, using participant surveys and distributor interviews to assess program influence on decision-making. The 2024 evaluation updated algorithms following a Net-to-gross Review.
Table 2: Instant Rebates Participant and Overall Free-ridership Algorithm 2022 Algorithm 2024 J Algorithm D7 the purchase of efficient lighting products. If your organization had not received the rebate and the cost for had been about $[RE...
AI summary Table 2 outlines the Instant Rebates Participant and Overall Free-ridership Algorithm, comparing the 2022 and 2024 algorithms. It includes questions about the likelihood of purchasing efficient lighting products without rebates and the calculation of a cost score (CS) for efficiency.
CUSTOM INCENTIVES PROGRAM Final Report 2024 DSM EVALUATION March 25, 2025
AI summary The document presents the Final Report of the 2024 Demand-Side Management (DSM) Evaluation for Nova Scotia's Custom Incentives Program, dated March 25, 2025. It includes visual elements but no substantive textual analysis or findings.
DEFINITIONS Accuracy Reflects the proximity of measurements to the true value. The number of years by which the first-year savings estimate is multiplied to obtain lifetime energy savings. This value takes into account variations in annual...
AI summary The document defines key terms related to energy efficiency program evaluations, including accuracy, lifetime energy savings, gross and net savings, evaluation plans, first-year savings, and the percentage of savings attributable to participants who would have taken similar actions without the program.
Table 1: Summary of 2024 Custom Incentives Program Evaluation Program Evaluation Type Methodology Component Impact Process Market Custom Comprehensive New Construction › Participant builder and non-participant modeller phone interviews (Ne...
AI summary The 2024 Custom Incentives Program Evaluation includes comprehensive assessments of the Custom and SEM programs. The evaluation methodology involves interviews, project reviews, and analysis of program participation and effectiveness. In 2023, SEM and EMIS were merged into a single program component.
Program Performance Table 2 below presents the participation levels, average net-to-gross ratios (NTGRs), evaluated gross and net savings at the generator, annual GHG emission reductions, as well as average EUL values for each program comp...
AI summary Table 2 provides data on program performance, including participation levels, net-to-gross ratios, gross and net savings, GHG emission reductions, and EUL values for various program components and Custom Incentives as a whole.
Custom General Key Findings and Recommendations 2024 Custom-Finding: Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in net peak demand savings at the generator in 2024, thereby surpassing the planned electrical en...
AI summary Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in peak demand savings in 2024, exceeding targets by 48% and 28%, respectively. Retrofit and Building-Optimization participation declined, while New Construction increased. Adjustments to savings included ratios of 1.012–1.068 across services. Free-ridership dropped to 15% for Retrofit and 28% for New Construction. Evaluated savings were 7% higher than E1's tracked data.
New Construction Key Findings and Recommendations 2024 New Construction-Finding: The demand savings calculation approach used by E1 for projects modelled in eQuest can generate results that are difficult to confirm. 2024 New Construction R...
AI summary The 2024 findings highlight issues with E1's demand savings calculation methodology in eQuest models, the need for M&V in large industrial projects, increased MURB participation in HRM, participant satisfaction, non-participant awareness, and decarbonization as a motivator. Recommendations include revising demand savings scripts and updating program eligibility criteria.
SEM Findings and Recommendations This subsection provides the key findings and recommendations from the SEM evaluation. The recommendations are also outlined in Appendix XVIII. 2024 SEM-Finding: SEM exceeded its net electrical energy and p...
AI summary SEM exceeded 2024 energy and peak demand savings targets by 6% and 14% respectively, maintained stable participation levels over five years, and improved M&V accuracy. Seven of 12 participants achieved savings, with peak demand estimates showing significant methodological improvements.
EfficiencyOne (E1), an independent, non-profit organization, is responsible for helping Nova Scotians improve the energy efficiency of their homes and workplaces by designing, marketing, and delivering demand-side management (DSM) for Nova...
AI summary EfficiencyOne (E1) is a non-profit organization that provides energy efficiency and demand response services in Nova Scotia through the Efficiency Nova Scotia (ENS) franchise. E1's 2024 DSM program portfolio includes the Custom Incentives program, which was evaluated by Econoler. The evaluation focuses on baseline definitions, savings calculation methodologies, parameter values, and net-to-gross ratios.
1 Custom Overview This section describes the Custom component of the Custom Incentives program, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Custom component of the Custom Incentives program, addresses past evaluation recommendations, and summarizes participation history within the program.
1.2 Follow-up on Past Evaluation Report Recommendations The Evaluator evaluated Custom in previous years and issued improvement recommendations. [Table](#page-174-1) 5 below provides a summary of the implementation status of each recommend...
AI summary The Evaluator reviewed Custom in previous years and provided improvement recommendations. Table 5 summarizes the implementation status of ongoing recommendations from the 2023 and 2022 Custom Incentives Evaluation Reports.
Table 5: Implementation Status of Past Recommendations for Custom # Recommendations Status Comments 2022 Retrofit R2 Require measurement and verification (M&V) efforts based on the International Performance Measurement and Verification Pro...
AI summary The document discusses the implementation status of past recommendations related to custom programs, including the use of M&V efforts based on the IPMVP Option C approach for retrofit projects, revisions to model review processes, monitoring free-ridership, and updating logic models for Custom Retrofit. Progress and completion statuses are outlined with specific comments.
New Construction In 2024, 28 projects claimed savings under New Construction: 27 complete projects and 1 project, a new industrial facility, that was partially completed and for which savings were claimed because the building was commissio...
AI summary In 2024, 28 New Construction projects achieved energy savings, including 27 completed projects and one partially completed industrial facility commissioned in 2024. Savings reached record highs, with 21.495 GWh and 4.004 MW in gross energy and peak demand savings. The number of completed projects increased after a decline between 2021–2023.
2 Custom Evaluation Approach The 2024 Custom evaluation comprised a comprehensive impact evaluation for Retrofit, Building Optimization, and New Construction. For Building Optimization, given its smaller contribution to Custom savings, NTG...
AI summary The 2024 Custom evaluation assessed Retrofit, Building Optimization, and New Construction programs, focusing on energy savings, GHG reductions, and participation analysis. NTGR 2021 data was applied for Building Optimization due to 2024 project inactivity. Objectives included calculating savings, analyzing sub-sector/region participation, and reviewing logic models.
Table 6: 2024 Custom Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › Are the gross savings calculated for a sample...
AI summary Table 6 outlines the 2024 Custom Evaluation Approach, focusing on calculating gross and net results, exploring opportunities to expand program presence, and updating program service logic. It includes research questions, methodologies, and evaluation objectives for various projects and programs.
Program Staff Interviews The New Construction Program Manager and Business Development Manager were interviewed to inform the Process evaluation. The interview guide is in Appendix X.
AI summary The New Construction Program Manager and Business Development Manager were interviewed as part of the Process evaluation. The interview guide is referenced in Appendix X of the document.
Tracking Sheet Audit Prior to performing any savings calculations, the Evaluator conducted an audit of the final 2024 tracking sheets to ensure they were complete and data entry was consistent. The detailed protocols used for the tracking...
AI summary An audit of the final 2024 tracking sheets was conducted to ensure completeness and consistent data entry. Audit protocols and results are detailed in Appendices IV and XIII.
Project File Reviews and Participant Site Visits or Follow-up Phone Interviews In the fall of 2024 and January 2025, Econoler and its subcontractor CDM Energy Solutions carried out a full technical review of project documentation for 19 re...
AI summary In fall 2024 and January 2025, Econoler and subcontractors conducted technical reviews of 19 Retrofit, 1 compressed air leak Retrofit, 2 Building Optimization, and 12 New Construction projects. Site visits and phone interviews were used to finalize reviews and collect free-ridership data for Retrofit projects, while New Construction reviews relied solely on project files. Appendices outlined protocols and interview guides for these processes.
Calculations Using Evaluation Results Building on all the above methods and collected data, the Evaluator calculated the first-year and lifetime energy and peak demand savings as per the calculation methodology presented in Sections [3,](#...
AI summary The Evaluator calculated first-year and lifetime energy and peak demand savings using methodologies outlined in Sections 3, 5, and 6. These calculations build on prior data collection and evaluation methods to quantify program impacts.
Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that if measurements were conduc...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations, excluding nonsampling errors. Margins of error were calculated for Retrofit and New Construction programs in the 2024 Custom evaluation, but not for Building Optimization due to full project reviews. Examples of calculations are in Appendix II of the DSM Programs Evaluation Executive Summary.
3.1 Tracking Sheet Audit To ensure service results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification and correctiv...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable service results. Corrective actions are detailed in Appendix IV, leading to corrected tracked savings results.
3.2.1 Savings Verification For compressed air leak audit and solar PV projects, the Evaluator verified savings by validating that the correct input parameters were used to estimate savings and applying a previously established adjustment r...
AI summary The Evaluator verified savings for compressed air leak audits and solar PV projects by validating input parameters and applying adjustment ratios. Energy and demand savings for air leak audits used 2021 Retrofit ratios, while solar PV projects used 2023 adjustment ratios.
3.2.2 Project Review Sampling Methodology For the regular Retrofit project category, the Evaluator used a stratified sampling approach to select 19 projects for review from a total of 44 projects completed in 2024. More specifically, the E...
AI summary The Evaluator used a stratified sampling approach to review 19 Retrofit projects from 44 completed in 2024, prioritizing larger projects and applying weighted average adjustment ratios. Two large compressed air leak audit projects were reviewed more thoroughly due to deviations from standard methodologies. Savings were extrapolated using adjustment ratios specific to solar PV estimation tools (Retscreen and PV Watts).
3.2.3 Project Review Findings The Evaluator reviewed the project sample to ensure the best measurement and verification practices were applied for commercial and industrial energy efficiency projects and adjusted the tracked savings accord...
AI summary The Evaluator reviewed energy efficiency projects to ensure best measurement and verification (M&V) practices were applied, adjusting tracked savings accordingly for commercial and industrial programs.
Regular Retrofit Project Reviews As a result of the review process, the Evaluator adjusted the savings of eight of the 19 sampled regular Retrofit projects. These are broken down as three projects having a combination of energy and peak de...
AI summary The Evaluator adjusted savings calculations for eight Retrofit projects, finding discrepancies in energy and peak demand estimates. Adjustments included recalculating insulation values, correcting pre-M&V vs. post-M&V data, and accounting for sporadic motor operation. The Evaluator recommends revisiting peak demand savings best practices for Retrofit projects to improve accuracy.
Compressed Air Leak Audit Project Review The Evaluator also conducted an in-depth review of one of two large compressed air leak audit projects completed by the same participant at two different facilities. These projects were conducted us...
AI summary The Evaluator reviewed a compressed air leak audit project using an innovative all-in-one platform, finding the process sound and reliable. Documentation and savings calculations were accurate, leading to an adjustment ratio of 1.0. Technical expertise from the OEM and participant staff ensured project quality, with no adjustments made to energy savings estimates.
3.3.2 Spillover For Retrofit, participant spillover occurs when participants implement eligible energy efficiency measures due to the influence of previous participation in the service without receiving any kind of additional support[13](#...
AI summary The text discusses spillover effects in the Retrofit program, where participants implement energy efficiency measures influenced by prior participation. The Evaluator used phone interviews and an algorithm to assess spillover, finding no measurable spillover for regular Retrofit participants. One respondent noted self-initiated measures, but savings were below 10,000 kWh.
4 New Construction Process Evaluation Results from the New Construction process evaluation are presented below. Subsection [4.1](#page-195-1) provides an overview of the evolution of service participation during 2019–2024. Subsections [4.2...
AI summary The New Construction Process Evaluation (2019–2024) assesses participation trends, awareness, motivations, barriers, and satisfaction. It examines market factors like building code updates and decarbonization, includes a jurisdictional scan of programs, and reviews the logic model for the initiative.
4.8.1 Program Objectives The program objectives stated in the Custom program manual that are most specific to the New Construction service are provided below. - 1) Influence electrical energy efficiency and system-peak demand reduction pro...
AI summary The New Construction service's program objectives include promoting energy efficiency, overcoming implementation barriers, and increasing awareness of high-efficiency design. The logic model inadequately addresses objectives related to barrier removal and awareness, and outcomes are not clearly tied to objectives. A revised logic model should align outcomes with program goals.
also a key barrier to highefficiency building design. Participating builders worry most about the financial aspects of building highefficiency buildings and how the costs compare to incentive levels. Lack of builder awareness: Lack of buil...
AI summary Key barriers to high-efficiency building design include financial concerns, lack of builder awareness about long-term savings and technology performance, insufficient energy modeller capacity, and high energy modelling costs. E1 highlights challenges in meeting eligibility requirements and limited technical support for builders.
4.8.4 Logic Model Summary To summarize, the evaluator identified four important barriers that the service is working to overcome. These barriers are addressed by and aligned with the program logic model and theory of change to varying degr...
AI summary The evaluator identified four barriers to the program's success: high energy modelling costs, high upfront costs, limited developer awareness, and insufficient modeller participation. While some barriers are addressed in the logic model, others are not. Updates to the model are recommended to align activities with all barriers, including adding high modelling costs and awareness gaps.
5 New Construction Impact Evaluation The objective of the 2024 New Construction impact evaluation was to determine gross and net electrical energy and peak demand savings. In 2024, two types of savings were claimed: (1) partial savings (pr...
AI summary The 2024 New Construction impact evaluation aimed to assess gross and net electrical energy and peak demand savings, distinguishing between partial savings (future projects) and final savings (2024 single-year projects).
5.1 Tracking Sheet Audit To ensure service results were reliably compiled, the Evaluator first performed a tracking sheet audit intended to verify the completeness and consistency of the data submitted by E1. The verification and correctiv...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of service results. Corrective actions are detailed in Appendix XIII, with adjusted tracked savings results presented in the report.
5.2.1 Sampling Methodology For the energy model review, the evaluator selected 12 of the 27 completed projects. The selected sample of 12 projects represented 68% of total tracked energy savings for projects completed in 2024. Only complet...
AI summary The evaluator sampled 12 of 27 completed projects, representing 68% of 2024 energy savings, excluding one incomplete industrial project. Gross savings were extrapolated using a weighted average adjustment ratio. The excluded project will be reviewed in 2025.
5.2.2 Project Review Findings The 12 project reviews were intended to validate the energy models developed for the baseline and proposed cases of each project and the resulting savings. The Evaluator based the review mainly on the project...
AI summary The evaluation of 12 projects confirmed robust energy model validation using detailed project files. EnergyPlus submissions require visual editor files for efficient review, and model weather files should be included in documentation. An industrial project with partial savings was addressed, with recommendations for M&V requirements in future industrial programs.
Energy Savings Positive or negative adjustments were made to the tracked gross energy savings of all 12 projects reviewed by the Evaluator for 2024. The energy model reviews resulted in an average adjustment ratio of 1.028 for gross energy...
AI summary Adjustments to energy savings for 12 projects in 2024 revealed discrepancies between modeled and actual installations, including HVAC system mismatches and incorrect ERV installations. An average adjustment ratio of 1.028 was applied, with improvements in as-built model accuracy due to E1's enhanced document control.
Peak Demand Savings In previous evaluations, to establish peak demand savings, the Evaluator extracted the hourly results of the revised building energy model and averaged the monthly peak demand reduction for the months of December, Janua...
AI summary The Evaluator and E1 used different methods to calculate peak demand savings, with minor discrepancies observed. The Evaluator adopted E1's eQuest script method for 2024 and recommends investigating discrepancies to ensure consistent calculation methodologies across software. Adjustment ratios showed an average of 1.003 with a 6.9% margin of error.
Gross Savings NTGR Net Savings Realization Rate Value Unit Value Value Unit Value Energy Savings Tracked Savings by E1 20.900 GWh 0.72 14.421 GWh Evaluation Results 21.495 GWh 0.72 15.476 GWh 107% Peak Demand Savings Tracked Savings by E1...
AI summary Evaluated energy and peak demand savings were higher than tracked savings due to adjustments in energy models and slightly lower free-ridership in 2024, which increased the realization rate.
6 Building Optimization Impact Evaluation The objectives of the 2024 Building Optimization impact evaluation were to determine gross and net electrical energy and peak demand savings, annually avoided GHG emissions, as well as EUL values a...
AI summary The 2024 Building Optimization Impact Evaluation assesses energy and peak demand savings, GHG emissions reduction, and EUL values. It categorizes savings into partial, single-year, and multiyear projects, with two single-year projects claiming final savings in 2024.
6.1 Tracking Sheet Audit To ensure service results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification and correctiv...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable service results. Corrective actions taken by the Evaluator are detailed in Appendix IV, leading to corrected tracked savings results presented in the report.
6.2.1 Project Review Findings The Evaluator reviewed both completed Building Optimization projects. Following the review, the Evaluator revised the energy savings of one project, and the demand savings of both projects. - › One project inv...
AI summary The Evaluator reviewed two completed Building Optimization projects, adjusting energy savings upward for one and demand savings downward for both due to miscalculations in original assessments. No average adjustment ratio was established, with total savings calculated by summing individual project evaluations. Partial savings claims will require future reviews.
Table 29: Comparison of 2024 Custom Tracked and Evaluated Savings at the Generator Gross Savings NTGR Net Savings Realization Rate Value Unit Value Value Value Unit Energy Savings Tracked Savings by E1 26.115 GWh 0.79 20.719 GWh Evaluation...
AI summary Table 29 compares tracked and evaluated energy and peak demand savings from 2024 custom programs. It highlights differences between gross and net savings, along with realization rates, showing that evaluation results often exceed tracked savings, with realization rates ranging from 90% to 109%.
8 Custom Key Findings and Recommendations The main objectives of the 2024 Custom evaluation were as follows: - › Calculate gross and net results, namely electrical first-year and lifetime energy savings, peak demand savings, as well as avo...
AI summary The 2024 Custom evaluation aimed to calculate energy savings, GHG emissions, and collect perspectives on New Construction participation motivations and barriers. Findings and recommendations are provided, with general and service-specific insights outlined in Appendix XIV.
General Custom Key Findings and Recommendations 2024 Custom - Finding: Custom net electrical energy and peak demand savings surpassed targets in 2024. Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in net peak dem...
AI summary Custom program exceeded 2024 energy and peak demand savings targets by 48% and 28%, respectively. Participation decreased in Retrofit and Building Optimization but increased in New Construction. Evaluator adjustments led to varying ARs across services. Free-ridership dropped for Retrofit and New Construction. Evaluated savings were 8-5% higher than E1's tracking.
New Construction Key Findings and Recommendations 2024 New Construction-Finding: The demand savings calculation approach used by E1 for projects modelled in eQuest can generate results that are difficult to confirm. The Evaluator found min...
AI summary The Evaluator identified issues with E1's demand savings calculation approach for eQuest-modeled projects, noted discrepancies in industrial project savings claims, and observed increased MURB participation. Recommendations include revising E1's methodology for consistency, requiring M&V for large industrial projects, and aligning with market trends.
9 SEM Overview This section describes the Strategic Energy Management (SEM) program component, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Strategic Energy Management (SEM) program, addresses past evaluation recommendations, and provides an overview of participation history within the program.
9.2 Follow-up on Past Evaluation Report Recommendations The Evaluator evaluated SEM in previous years and issued improvement recommendations. [Table](#page-31-0) 30 provides a summary of the implementation status of the recommendations pre...
AI summary The Evaluator assessed SEM in previous years and provided improvement recommendations. Table 30 summarizes the implementation status of these recommendations from the 2023 SEM Evaluation Report.
Table 31: 2024 SEM Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › Are the gross savings for each project accuratel...
AI summary Table 31 outlines the 2024 SEM Evaluation Approach, focusing on calculating gross and net results from energy management initiatives. It includes evaluation objectives, research questions, and methodologies such as tracking sheet audits, project file reviews, and calculation of energy savings and GHG emissions.
Project File Reviews In January 2024, a total of seven project reviews were conducted, one for each participant that generated savings. Of these project reviews, three were supported by phone interviews, while site visits were conducted fo...
AI summary In January 2024, seven project reviews were conducted, with three involving phone interviews and one requiring a site visit by Equilibrium Engineering. Econoler handled desk reviews and interviews, while the Evaluator coordinated with the Service Provider for clarifications. The methodology is detailed in Appendix XVI.
11 SEM Impact Evaluation The objectives of the 2024 SEM comprehensive impact evaluation were to determine project gross and net electrical energy and peak demand savings as well as annually avoided GHG emissions, EUL values, and associated...
AI summary The 2024 SEM comprehensive impact evaluation aimed to assess project energy and peak demand savings, GHG emissions reductions, EUL values, and lifetime electrical energy savings from Strategic Energy Management initiatives.
11.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The results obtaine...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, resulting in corrected tracked savings figures. The audit findings are detailed in Appendix XV.
11.2 Gross Savings For SEM, gross savings correspond to the change in energy consumption resulting from actions taken by participants regardless of their reasons for participating.[39](#page-36-1) The subsections below provide a descriptio...
AI summary Gross savings for Strategic Energy Management (SEM) are defined as changes in energy consumption from participant actions, irrespective of motivation. The section outlines methodology for reviewing seven 2024 SEM projects, including assessments of interactive effects, Effective Useful Life (EUL) values, and revised electrical savings.
11.2.1 Project Review Findings The Evaluator reviewed the calculation methodologies for the seven projects based on project documentation and information from interviews with participants and the Service Provider. All but two projects used...
AI summary The Evaluator reviewed seven projects' savings calculation methodologies, noting most used a bottom-up M&V approach consistent with Custom Retrofit standards. A top-down approach was deemed appropriate for whole-facility measures where component-level monitoring was impractical. The Service Provider's methodology selection aligned with approved M&V procedures.
11.2.2 Interactive Effects For each project reviewed, the Evaluator validated if interactive effects should have been considered and concluded that no interactive effects impacted the concerned measures. Hence, no additional interactive ef...
AI summary The Evaluator assessed whether interactive effects should be considered for each project but concluded none impacted the measures. Consequently, no additional interactive effects adjustments were applied to measured savings.
11.3.1 Evaluated Net Savings Since spillover and free-ridership effects were considered nil, net SEM impacts are equal to the gross savings generated by the program component. The 2024 SEM net electrical energy and peak demand savings were...
AI summary The 2024 SEM program achieved 4.478 GWh of net electrical energy savings and 0.538 MW of peak demand reductions, exceeding targets by 6% and 14% respectively. These savings corresponded to 2,114 tonnes of annual CO2 eq GHG reductions, with no spillover or free-ridership effects considered.
12 SEM Key Findings and Recommendations As mentioned previously, the main objectives of the 2024 SEM evaluation were as follows: › Calculate SEM gross and net results, namely electrical first-year and lifetime energy savings, peak demand s...
AI summary The 2024 SEM evaluation found that SEM exceeded its net electrical energy and peak demand savings targets by 6% and 14%, respectively. Participation remained stable with 12 participants, and M&V methodologies showed improved accuracy, particularly for peak demand savings. The Evaluator noted no recommendations for the program.
Custom Appendix I: Retrofit Participant Interview Guide Appendix II: Retrofit Algorithm for Free-Ridership Calculation Appendix III: Retrofit Algorithm for Participant Spillover Calculation Appendix IV: Retrofit and Building Optimization T...
AI summary The document outlines appendices related to a 'Custom' regulatory process, including tools for evaluating energy efficiency programs. Key components include free-ridership and spillover calculation algorithms, participant interview guides, project review protocols, and adjustment ratio examples, emphasizing program evaluation and data collection methods.
2024 DSM EVALUATION March 26, 2025 In collaboration with:
AI summary The 2024 Demand-Side Management (DSM) Evaluation document outlines a collaborative effort involving unspecified partners, though specific content details are not provided in the text. The evaluation focuses on assessing DSM programs and their outcomes.
[ASK IF $ ≥0] - C5. As part of its Custom Retrofit program, Efficiency Nova Scotia gave your organization a $ incentive for the [Investigation or Feasibility] study. Without this incentive, what is the likelihood that you would have conduc...
AI summary The text asks respondents about the impact of financial incentives on their likelihood of conducting energy efficiency studies and implementing projects. It focuses on Efficiency Nova Scotia's Custom Retrofit program, evaluating how incentives influence decision-making and project timelines.
E. Spillover - E1. Since taking part in the Custom Retrofit program, have you implemented any additional energy efficiency measures outside of the program? - 1. Yes - 2. No [THANK AND TERMINATE] - 98. Don't know [THANK AND TERMINATE] - 99....
AI summary The survey explores whether participants in the Custom Retrofit program implemented additional energy efficiency measures outside the program, their financing sources, and the influence of the program on these decisions. It also requests details on measures and reasons for not using Efficiency Nova Scotia programs.
APPENDIX IV Retrofit and Building Optimization Tracking Sheet Audit This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the eva...
AI summary This appendix details the results of a tracking sheet audit conducted by the Evaluator to verify the accuracy and completeness of data submitted by EfficiencyOne (E1) for the Retrofit and Building Optimization programs. The audit ensured consistency in calculation methods and parameters used to determine energy and peak demand savings.
Project Review Protocol The Evaluator used the same project review protocol in 2024 as the one used for the 2023 Custom Retrofit evaluation. The protocol includes questions and assessment fields for measurement and verification (M&V) plans...
AI summary The Evaluator employed a consistent project review protocol in 2024, mirroring the 2023 Custom Retrofit evaluation. This protocol includes M&V plan assessments, measure-specific worksheets, and pre-review documentation analysis from EfficiencyOne (E1), covering feasibility studies, savings calculations, and M&V reports.
Efficency Nova Scotia Measure Review Protocol - 2024
AI summary The document presents the Efficency Nova Scotia Measure Review Protocol for 2024, outlining procedures for reviewing energy efficiency measures. It is part of a regulatory proceeding related to energy efficiency programs and their evaluation.
Revised Savings Calculation The Evaluator found that the assumptions and the analysis performed by E1 were generally sound. Moreover, after reviewing the savings calculations, the Evaluator found mistakes in some of the Excel formulas used...
AI summary The Evaluator found E1's analysis generally sound but identified errors in refrigeration load formulas using incorrect target temperatures, leading to downward adjustments in consumption and demand savings. The Effective Useful Life (EUL) was also revised from 15 to 11 years due to varying unit lifespans from mixed refrigerant upgrades.
Adjustment Ratio Calculation Adjustment ratios are determined by comparing revised savings values with tracked savings values. Due to the revisions made to this project, the calculated adjustment ratio for energy savings was 0.825. $$\frac...
AI summary Adjustment ratios are calculated by comparing revised energy savings (105,425 kWh) to tracked savings (127,826 kWh), resulting in a ratio of 0.825 due to project revisions. The formula is provided as (Revised ES / Tracked ES) = 0.825.
Introduction – Online Survey Narrative Research and Econoler are currently conducting a formal evaluation of the Efficiency Nova Scotia Custom New Construction program. Your organization recently entered into an agreement in [CPA ACCEPTED...
AI summary Narrative Research and Econoler are evaluating Efficiency Nova Scotia's Custom New Construction program. The survey seeks to understand participants' decisions in building energy-efficient 'better-than-code' buildings, including material and equipment choices exceeding code requirements.
D. Cross-Influence - D1. [SINGLE RESPONSE] Before participating in the Custom New Construction program for the [INSERT PROJECT NAME] project, had your organization already participated in Custom New Construction or in another Efficiency No...
AI summary The document outlines a survey assessing cross-influence effects of Efficiency Nova Scotia programs on new construction projects. It evaluates whether prior participation in energy efficiency programs or exposure to promotional materials influenced technical evaluations and cost-effectiveness considerations for building projects.
APPENDIX VIII New Construction Participant Interview Guide (Completed Projects)
AI summary This document outlines an interview guide for participants in completed new construction projects under a Nova Scotia regulatory proceeding, focusing on data collection for program evaluation and stakeholder engagement.
Table 3: Overview of Data Collection Activity Descriptor This Instrument Instrument Type Semi-directed in-depth interview Estimated Time to Complete 15-20 Minutes Target Audience › Participant Builders Expected Number of Completions › Part...
AI summary This document outlines data collection activities involving semi-directed in-depth interviews with 12 participant builders to assess free-ridership levels, program satisfaction, and barriers to participation in New Construction projects. Econoler is responsible for fielding the interviews, with an estimated timeline of November 2024.
C. Cross-Influence - C1. Before participating in the Custom New Construction program for the building we discussed today, had you already taken part in Custom New Construction or another Efficiency Nova Scotia program? - 1. Yes, Custom New...
AI summary The text outlines survey questions assessing whether prior participation in Efficiency Nova Scotia programs or exposure to promotional materials influenced decisions in new construction projects, focusing on cross-influence effects on energy efficiency choices and cost evaluations.
- b. If less than 8, please explain the reason(s) for your score. Aspects of the program [READ AND RANDOMIZE] Score Reason 1. The overall program Response 98 DK99 Ref _97 N/A 2. The responsiveness of Efficiency Nova Scotia employees to you...
AI summary The text includes a survey assessing the Efficiency Nova Scotia program, covering aspects such as program responsiveness, approval speed, and value. It also explores challenges faced and suggestions for improvement. Additionally, it addresses the importance of reducing GHG emissions and the influence of electrification on participation in construction programs.
Table 7: Overview of Data Collection Activity Descriptor This Instrument Instrument Type Semi-directed in-depth interview Estimated Time to Complete 45 min for Program Manager, 45 min for BDM Target Audience E1 program staff: › Commercial...
AI summary This document outlines data collection activities for a program evaluation, focusing on interviews with program staff to explore opportunities for expanding participation and updating service logic. Research questions address participation trends, barriers, and relationships with modelers.
F. Challenges and Opportunities - F1. Overall, what is the main strength of the New Construction program? - F2. What is the main challenge of the New Construction program? - F3. What feedback have you heard from participants about the New...
AI summary The section outlines a series of questions aimed at evaluating the New Construction program, focusing on its strengths, challenges, participant feedback, and potential improvements. It seeks insights into program effectiveness, areas for enhancement, and future directions.
G. Program evaluation G1. [Program manager only] We will look at Commercial new construction program eligibility criteria in other jurisdictions to inform the process evaluation. Are there specific jurisdictions you recommend we consider?...
AI summary The document outlines a request to evaluate eligibility criteria for Nova Scotia's Commercial New Construction program by referencing other jurisdictions, and invites additional input on the New Construction program. The focus is on program evaluation and stakeholder feedback.
Modelling Review Checklist
AI summary A checklist for reviewing models in a Nova Scotia regulatory proceeding, likely related to energy efficiency, demand-side management, or program evaluation. The document outlines criteria for assessing model accuracy, assumptions, and alignment with regulatory goals.
Reminder of Evaluation Goals and Key Principles While developing the checklist, the Evaluator kept in mind the key evaluation goals and the five guiding principles presented in the 2020-2022 Overall Strategic Evaluation Plan. Notably for N...
AI summary The evaluation process for complex programs like Custom New Construction focuses on measures with the highest savings impact, avoiding comprehensive reviews of entire models or M&V procedures as per Principle 4 of the 2020-2022 Strategic Evaluation Plan.
Table 1: Participant Interview Questionnaire and Free-ridership Algorithm Question Answer Score A1 We hope to interview the key decision-makers that played a key role in the decision to build a better than-code building. Were you a key 1)...
AI summary This table outlines a participant interview questionnaire focused on identifying key decision-makers involved in building better-than-code buildings, with specific questions about their roles and other potential decision-makers.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix details the results of a tracking sheet audit conducted by the Evaluator to verify the completeness and accuracy of data submitted by E1. The audit ensured consistency in parameters used to calculate program results and validated calculation steps.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and filled out in the tracking sheet submitted...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to verify data completeness and accuracy in EfficiencyOne's submissions, focusing on consistency of parameters and calculation methods for program results.
APPENDIX XVI SEM Project Review Protocol The SEM project review protocol used for the 2024 evaluation was the same protocol used in 2023. It includes sections that served to review both bottom-up and top-down approaches that were both used...
AI summary The 2024 SEM Project Review Protocol reused the 2023 protocol, incorporating bottom-up and top-down evaluation approaches. Evaluations involved phone interviews/site visits after reviewing EfficiencyOne digital files, with a net-to-gross ratio (NTGR) of 1.00 from the 2023 evaluation applied to the current DSM cycle.
Evaluation Approach The purpose of the 2024 evaluation was to calculate gross and net results of the program components, namely electrical first-year and lifetime energy savings, peak demand savings, as well as avoided greenhouse gas (GHG)...
AI summary The 2024 evaluation aimed to calculate gross and net results of program components, including electrical first-year and lifetime energy savings, peak demand savings, and avoided greenhouse gas emissions. Table 1 outlines the types of evaluations conducted and their corresponding methodologies.
Table 1: Summary of 2024 Direct Installation Program Evaluation Program Eva aluation Type Mothodology Component Impact Process Market Market Methodology Small Business Energy Solutions Condensed - - Tracking sheet audit Unitary savings rev...
AI summary The table summarizes the 2024 Direct Installation Program Evaluation, focusing on the Small Business Energy Solutions program. It outlines the evaluation methodology, including tracking sheet audits, savings reviews, and GHG emission reduction calculations.
SBES Findings and Recommendations This subsection presents the key findings from the SBES evaluation. 2024 SBES-Finding: SBES surpassed its net electrical energy savings by 2% and fell short of its net peak demand savings target by 16%. 20...
AI summary The 2024 SBES evaluation found that the program exceeded energy savings targets by 2% but missed peak demand goals by 16%. Participation rose by 53%, with DIY being the dominant route. Non-participant spillover was nil, and discrepancies were noted between evaluator and E1's tracked savings.
1 SBES Overview This section describes the Small Business Energy Solutions (SBES) program component, follows up on past evaluation recommendations, and provides an overview of participation history.
AI summary This section outlines the Small Business Energy Solutions (SBES) program, addresses past evaluation recommendations, and reviews participation history within the program.
1.2 Follow-up on Past Evaluation Report Recommendations The Evaluator evaluated SBES in previous years and issued improvement recommendations. [Table](#page-133-2) 5 below provides a summary of the implementation status of past recommendat...
AI summary The Evaluator reviewed past recommendations for the SBES program and notes that all remaining recommendations are currently in progress. A table summarizes the implementation status of these recommendations.
Table 5: Implementation Status of Past Recommendations for SBES # Past Recommendations Status Comments 2023- SBES-R1 Consider improvements to the Customer Information System, notably by reviewing how missing entries are treated and extract...
AI summary Table 5 outlines the implementation status of past recommendations for the Small Business Energy Solutions (SBES) program. Two recommendations are discussed: improving the Customer Information System and implementing communication strategies for participants nearing project expiration. The first is in progress, while the second has been completed.
Table 6: 2024 SBES Evaluation Approach Evaluation Objectives Research Questions Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › What are the formulas and savings parameters for...
AI summary Table 6 outlines the 2024 SBES Evaluation Approach, detailing objectives, research questions, and methodologies for evaluating the Small Business Energy Solutions program. It includes calculating gross and net results, collecting non-participant perspectives, and conducting a tracking sheet audit.
GHG Emission Reduction Calculation To obtain net avoided GHG emissions in CO2 eq for SBES, the Evaluator multiplied the net energy savings by the latest Nova Scotia-specific factor for GHG emissions generated by electricity production. Thi...
AI summary The Evaluator calculated net avoided GHG emissions for SBES by multiplying net energy savings by Nova Scotia-specific electricity production GHG factors derived from NS Power data.
Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were condu...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations. For the 2024 SBES evaluation, no margins were calculated, though 2023 free-ridership results include margins. Confidence levels exclude non-sampling errors like data entry or response biases.
3.3 Past Energy Efficiency Projects Among Small Businesses
AI summary This section reviews past energy efficiency initiatives targeting small businesses in Nova Scotia, highlighting programs like SBES (Small Business Energy Solutions) and BER (Business Energy Rebates) administered by ENS (Efficiency Nova Scotia). It discusses outcomes such as cost savings, GHG reductions, and challenges including participation rates and program evaluation complexities.
4 SBES Impact Evaluation The objectives of the 2024 SBES impact evaluation were to determine gross and net electrical energy and peak demand savings.
AI summary The 2024 SBES impact evaluation aimed to assess gross and net electrical energy savings and peak demand reductions. The evaluation focused on quantifying the program's effectiveness in achieving energy efficiency outcomes for small businesses.
4.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The results obtained...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of program results. Corrected tracked savings validated by the Evaluator are referenced in Appendix III.
4.2.1 Unitary Energy and Peak Demand Savings Savings for SBES Audit and DIY projects are established through calculations using data specific to each project. The 2024-2025 DSM MA provides a detailed description of inputs, references, and...
AI summary The 2024-2025 DSM MA outlines changes to unitary savings parameters for SBES measures, including updates to SEER, EER, and PCF values. The Evaluator validated these changes for top measures, with other parameters unchanged. The CIRx tool provides methodology details.
4.3.1 Free-Ridership For SBES, free-ridership occurs when participants would have implemented energy efficiency upgrades in the absence of the program component. The free-ridership levels for DIY and Audit projects were assessed during the...
AI summary The document discusses free-ridership in the context of the SBES and CDI pilot programs. Free-ridership refers to participants who would have implemented energy efficiency upgrades without the program. For SBES, free-ridership levels from 2023 were used in 2024 as no new data was collected. For the CDI pilot, no free-ridership activity was conducted in 2023, leading to an assumption of no free-ridership.
4.3.3 Non-Participant Spillover Using the approach described in Section 2, the Evaluator assessed non-participant spillover from a sample of 30 commercial customers. Respondents were asked if they had implemented energy efficiency projects...
AI summary The Evaluator assessed non-participant spillover by surveying 30 commercial customers, finding that 14 implemented energy efficiency measures without E1's assistance, with minimal influence from E1's actions. The non-participant spillover level was determined to be nil.
4.3.4 Net-to-gross Ratio Calculation The NTGR results from the comprehensive impact evaluation performed in 2023 were used for the 2024 evaluation. The NTGR for SBES was calculated using the following equation. NTGR = (1 – % Free-ridership...
AI summary The Net-to-gross Ratio (NTGR) for the Small Business Energy Solutions (SBES) program was calculated using the equation NTGR = (1 – % Free-ridership + % Participant Spillover), based on the 2023 comprehensive impact evaluation results used for the 2024 evaluation.
Table 14: 2024 SBES NTGRs Project Path Free-ridership Participant Spillover NTGR DIY 20% 0% 0.80 Audit 12% 0% 0.88 CDI Pilot - - 1.00 4.3.5 Evaluated Net Savings
AI summary Table 14 presents 2024 SBES NTGRs for different project paths, including DIY, Audit, and CDI Pilot. The table shows free-ridership, participant spillover, and NTGR values. Section 4.3.5 discusses evaluated net savings.
5 SBES Key Findings and Recommendations The main objectives of the 2024 SBES evaluation were as follows: - › Calculate gross and net SBES results, namely electrical first-year and lifetime energy savings, peak demand savings, as well as av...
AI summary The 2024 SBES evaluation found that the program met its net electrical energy savings target (10.854 GWh, 2% over target) but missed its peak demand savings target (2.155 MW, 16% short). Evaluated savings exceeded E1's tracked results by 4% and 6% respectively. Participation rose 53% year-over-year, with DIY Path dominating. Spillover surveys found no non-participant impact.
This appendix presents the main results of the tracking sheet audit performed by the Evaluator, which was aimed at: - › Verifying that all data fields required for the evaluation were included and properly filled out in the tracking sheet...
AI summary This appendix outlines the results of a tracking sheet audit conducted by the Evaluator to ensure data accuracy and consistency in EfficiencyOne's submitted information, including verification of parameters used for calculating program results.
DEMAND RESPONSE PROGRAM Final Report 2024 DSM EVALUATION March 25, 2025
AI summary The Final Report from the 2024 DSM Evaluation, dated March 25, 2025, provides an assessment of the Demand Response Program. It focuses on evaluating the effectiveness of demand-side management initiatives, including program performance, cost recovery, and alignment with regulatory objectives.
Table 1: Summary of Demand Response Program Evaluation Evaluation Type Methodology Program Component Process Market Impact Residential Demand Response Comprehensive › Tracking sheet audit › Whole-house consumption data analysis › Measure A...
AI summary The document evaluates the performance of demand response programs, including residential and BNI demand response, using methodologies such as tracking sheet audits, whole-house consumption data analysis, and calculations based on evaluation results. Table 2 provides participation levels and evaluated capacity for these programs.
Residential DR Findings and Recommendations This subsection presents the key findings and recommendations from the Residential DR evaluation. 2024 Res DR-Finding: In 2024, Residential DR new and total available DR capacities both amounted...
AI summary The 2024 Residential DR evaluation found underperformance in achieving planned DR capacity (0.057 MW vs. 0.210 MW). Mysa thermostats dominated (83% of devices), but preheating errors reduced capacity. Whole-house data regression models were used for baseline calculations, with plans to refine unitary savings metrics. A 2025 recommendation calls for re-analyzing Eco Shift Pilot data to improve accuracy.
BNI DR Findings and Recommendations This subsection presents the key findings and recommendations from the BNI DR evaluation. 2024 BNI DR - Finding: Total available DR capacity at the generator amounted to 8.034 MW that included 5.669 MW o...
AI summary The 2024 BNI DR evaluation found that the program exceeded its 7.030 MW capacity target (8.034 MW achieved) with 76 participants, a 700% growth from 2023. While E1's guidelines were generally effective, gaps in baseline considerations were identified. Recommendations include updating documentation for building operating hours, lookback windows, and AMI data exclusions, and continuing project reviews for capacity evaluation.
INTRODUCTION EfficiencyOne (E1), an independent, non-profit organization, is responsible for helping Nova Scotians improve the energy efficiency of their homes and workplaces by designing, marketing, and delivering demand-side management (...
AI summary EfficiencyOne (E1) manages Nova Scotia's demand-side management (DSM) programs, including a demand response (DR) program evaluated in 2024. The DR program's available capacity is measured based on potential load reduction during winter events (Dec-Feb), excluding weekends/holidays, with capacity calculated per participant over two hours of DR events.
1 Residential DR Overview This section describes the Residential DR program component, follows up on past evaluation recommendations, and provides an overview of Residential DR participation history.
AI summary This section outlines the Residential Demand Response (DR) program, addresses past evaluation recommendations, and reviews participation history. It emphasizes program evolution and historical engagement data.
3 Residential DR Impact Evaluation The objective of the 2024 Residential DR impact evaluation was to determine new and total available DR capacities.
AI summary The 2024 Residential DR Impact Evaluation aimed to assess new and total available demand response (DR) capacities, focusing on residential programs. This evaluation seeks to quantify the potential for DR participation and its implications for grid management and energy efficiency initiatives in Nova Scotia.
3.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. It should be noted t...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, excluding Residential DR due to pre-established methodology. The audit ensured necessary information was provided for savings calculations, with corrective actions detailed in Appendix I.
3.2 Demand Response Capacity For Residential DR, the metric is the available DR capacity that corresponds to the load reduction made available for peak demand events in participant homes. For the Eco Shift Pilot pathway, the Evaluator esta...
AI summary The document discusses residential demand response (DR) capacity metrics, noting that the Eco Shift Pilot pathway used a unitary DR capacity value multiplied by participating thermostats. While new and total capacities are reported separately, they are equivalent in the first year. However, low EV participation in 2024 prevented establishing a unitary DR capacity for that category.
Unitary Available DR Capacity As outlined in [Table](#page-8-0) 10 above, the average available DR capacity across all events was 177 W per household. The household data included in the metering analysis were used to establish an average o...
AI summary The Evaluator calculated a unitary available DR capacity of 39.4 W per thermostat for 2023-2024, based on household data. A higher expected capacity of 57.7 W/thermostat was proposed if preheating errors are resolved. The Evaluator recommends re-analyzing Eco Shift Pilot data in 2025 to confirm improved capacity, though the higher value was not used for 2023-2024 savings calculations.
3.2.3 Interactive Effects In a home, interactive effects occur when the implementation of energy efficiency products has an impact on the energy consumption of other elements such as heating and cooling. For the Eco Shift Pilot pathway, al...
AI summary Interactive effects occur when energy efficiency measures impact other home systems like heating/cooling. The Eco Shift Pilot evaluates these effects using whole-house metering data (AMI) over device-level data, capturing interactions with uncontrolled systems (e.g., heat pumps) and offsetting thermostat load reductions.
3.2.4 Effective Useful Life Although no electrical energy savings are expected from DR initiatives, the Evaluator established an EUL value since the available DR capacity can persist over time. For the Eco Shift Pilot pathway, an EUL value...
AI summary The Evaluator assigns an Effective Useful Life (EUL) of one year to Demand Response (DR) capacity in the Eco Shift Pilot pathway, as participation includes all active participants and lacks data on program longevity. This avoids extrapolating capacity over a participant's lifetime, ensuring accurate evaluation of available DR resources.
4 Residential DR Key Findings and Recommendations As mentioned previously, the main objective of the 2024 Residential DR evaluation was as follows: › Calculate Residential DR results, namely new and total available DR capacities This secti...
AI summary The 2024 Residential DR evaluation found that targets were unmet, with 0.057 MW achieved versus 0.210 MW planned. Mysa thermostats dominated (83% of devices), but preheating errors reduced capacity. A regression model using whole-house data calculated DR capacity, and a 2025 Eco Shift Pilot data analysis is recommended to improve accuracy.
5 BNI DR Overview This section describes the Business, Non-profit, and Institutional (BNI) Demand Response (DR) program component, follows up on past evaluation recommendations, and provides an overview of BNI DR participation history.
AI summary This section outlines the Business, Non-profit, and Institutional (BNI) Demand Response (DR) program, addressing past evaluation recommendations and summarizing participation history. It provides context for the program's structure and evolution based on prior assessments.
Participation in Events Based on the review of 30 projects, it was found that participants did not participate in events around 31% of the time. In conducting project reviews, participants with a lower available DR capacity were found to h...
AI summary Analysis of 30 projects revealed 31% non-participation in events, with lower DR capacity correlating to fewer participations. Non-participation included no demand reduction or events during non-operational hours. Extrapolated participation averaged 64 meters per event. Projects were categorized for morning/evening events, with 74 suited for mornings and 19 for evenings, influencing event calling strategies.
Table 15: 2024 BNI DR Evaluation Approach Evaluation Objectives Research Questions Methodology Establish available DR capacity results for the DR Aggregator pathway › Are the data in the tracking sheet complete, accurate, and consistent? ›...
AI summary This document outlines the evaluation approach for the 2024 BNI Demand Response (DR) program, focusing on assessing the completeness, accuracy, and consistency of data in the tracking sheet, as well as the application of measurement and verification (M&V) methodologies.
Project Reviews with Meter Data Analysis E1 staff sampled and reviewed a total of 30 projects to establish tracked available DR capacity. The sample was stratified so that the 20 projects generating the largest amount of tracked available...
AI summary E1 staff reviewed 30 stratified DR projects to assess tracked available DR capacity, validating adjustments using updated M&V methodology. The review aimed to evaluate DR capacity and confirm the appropriateness of new M&V rules for BNI DR projects, with results detailed in Subsection 3.2.1.
Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...
AI summary The Evaluator aimed for a 10% margin of error at 90% confidence for sample-based evaluations, excluding non-sampling errors. For BNI DR, adjustment ratios were analyzed with calculated margins of error. Non-sampling errors like data entry or response inaccuracies were not included in confidence level calculations.
7.1 Tracking Sheet Audit To ensure program component results were reliably compiled, the Evaluator first performed a tracking sheet audit aimed at verifying the completeness and consistency of the data submitted by E1. The verification and...
AI summary The Evaluator conducted a tracking sheet audit to verify the completeness and consistency of data submitted by E1, ensuring reliable compilation of program results. Corrective actions are detailed in Appendix VI, leading to the presentation of corrected tracked available DR capacity in the report.
7.2.1 Project Reviews and Meter Data Analysis For the Aggregator pathway, available DR capacity is established by comparing the predicted loads (baseline) to the actual loads during DR events using meter data. Baselines for DR events are e...
AI summary The document outlines E1's methodology for calculating available DR capacity using meter data and adjustment factors, including a shift from scalar to additive adjustments. It describes project reviews of 30 projects, stratified sampling, and evaluation processes guided by the BNI DR Baseline Considerations document to verify compliance and assess new rules.
Project Review Findings The most frequent adjustment made by E1 to the available DR capacity calculation was to set the available DR capacity to zero due to non-participation in events. If no obvious load shed was observable for an event w...
AI summary E1 frequently adjusted DR capacity calculations by setting values to zero due to non-participation, incorrect metering data, or operational hour mismatches. The Evaluator agreed with these adjustments but recommended explicitly documenting criteria for zeroing DR capacity and ensuring consistent lookback window adjustments. Overall, E1 project reviews were deemed appropriate.
7.2.3 Effective Useful Life Although no energy savings were expected under the Aggregator pathway, the Evaluator established an effective useful life (EUL) value to express for how many years available DR capacity might persist, i.e. as lo...
AI summary The Evaluator assigned an Effective Useful Life (EUL) of one year to DR capacity under the Aggregator pathway, as participation includes all active participants annually. No extrapolation over lifetime is needed due to lack of data on long-term participation, and the average number of winters subscribed should be used for new DR capacity.
8 BNI DR Key Findings and Recommendations As mentioned previously, the main objective of the 2024 BNI DR evaluation was as follows: › Calculate BNI DR results, namely new and total available DR capacities This section provides the Evaluato...
AI summary The 2024 BNI DR program exceeded its total available DR capacity target (7.030 MW) with 8.034 MW achieved, including 5.669 MW of new capacity. Participation increased 700% (9 to 76 participants), but average capacity per participant dropped from 263 kW to 106 kW. E1's calculation guidelines were mostly appropriate but had gaps identified by the Evaluator.
Form of the Regression After reviewing existing literature to identify the most appropriate baseline methodology for this evaluation, the Evaluator decided to use a regression model that considers the time of week and outdoor temperature t...
AI summary The Evaluator chose a regression model incorporating time-of-week and outdoor temperature to establish a baseline for evaluating a smart thermostat program. This approach accounts for temperature impacts on electricity consumption and leverages a large dataset, contrasting with methods using previous similar days. The model's preference stems from its common use in similar programs and ability to differentiate temperature impacts across varying times of the week.
Where: - $\rightarrow$ $\beta_{D,H}$ is the regression intercept. - $\alpha_{D,H}$ is the regression slope. - $\rightarrow$ RMSE h is the hourly model root mean square error. - $n_h$ is the number of observations. - $\rightarrow \bar{x}_h$...
AI summary The text outlines statistical methods for calculating uncertainty in demand response (DR) load reduction measurements, including equations for standard error propagation, root mean square error (RMSE), and HDD-based evaluation frameworks to quantify DR capacity accuracy.
Residential DR: Event Day Graphs of Expected Vs Actual Loads 0 5 10 15 20 25 0 0.5 1 1.5 2 2.5 3 3.5 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Load (kW) Hour of Day Event Hour Tag Expected Load (kW) Actual Load (kW) H...
AI summary The document presents event day graphs comparing expected versus actual residential demand response (DR) loads, highlighting discrepancies between projected and real-world energy consumption during specific events. Visual data illustrates load (kW) by hour of day, including HDD15 metrics, to evaluate DR program performance.
Test and Validate Test the lookback window against past DR events to validate its effectiveness.
AI summary The text instructs testing the lookback window against historical Demand Response (DR) events to assess its effectiveness in validating program performance or outcomes.
2024-2025 DSM MEASURE ASSESSMENT Final Report 2024 EVALUATION EDITION March 25, 2025
AI summary The 2024-2025 DSM Measure Assessment Final Report evaluates demand-side management initiatives in Nova Scotia. It is presented as the 2024 Evaluation Edition, dated March 25, 2025, though no detailed content is provided in the excerpt.
Development and Review Process Savings are established using one or more of the following approaches: Literature reviews of TRMs; metering studies and evaluation reports; engineering calculations; adjustments based on data collected throug...
AI summary The document outlines methods for establishing savings in the Measure Assessment (MA), including data collection approaches and parameter calculation guidelines. It details the use of three-year averages for most parameters, exceptions for rapidly changing data, and updates to specific parameters in the 2024 MA. New measures added in 2024 are also highlighted.
Electrical Energy Savings The following equation is used to calculate electrical energy savings. (ℎ) - = (ℎ) × (1 - + (%)) × " " " - − " (%) 1 The average retired appliance efficiency levels and sizes are likely to evolve overtime as the y...
AI summary The text provides an equation for calculating electrical energy savings, factoring in appliance efficiency and size changes over time. It references the 2024-2025 DSM Measure Assessment, noting that retired appliance efficiency levels evolve as manufacturing years become more recent.
Verification of Climate Data Validity Considering that the 1992 ADS study remains the only well-documented study for cold climates, it is still used to establish interactive effects. To ensure that the findings of the ADS study are valid a...
AI summary The document compares Nova Scotia's climate data with Quebec's (Trois-Rivières) to assess the validity of the 1992 ADS study for calculating interactive effects in energy programs. It concludes that while heating seasons differ by 6% and cooling seasons by 22%, these differences are deemed insignificant for applying the ADS study's findings to Nova Scotia.
APPENDIX V Net-to-Gross Ratio Review
AI summary This appendix reviews the Net-to-Gross Ratio (NTGR), a metric used to evaluate the effectiveness of energy efficiency programs by comparing actual energy savings to projected savings. The analysis considers factors such as program design, participant behavior, and measurement methodologies to assess program performance and compliance with regulatory standards.
EXECUTIVE SUMMARY EfficiencyOne (E1) commissioned Econoler to evaluate E1's 2023-2025 DSM program portfolio. As part of the evaluation scope, E1 asked Econoler to present a review of net-to-gross ratio (NTGR) best practices and update the...
AI summary EfficiencyOne commissioned Econoler to evaluate its 2023-2025 DSM program portfolio, focusing on updating net-to-gross ratio (NTGR) best practices and free-ridership methodologies. The review relied on Uniform Methods Project (UMP) guidelines and regional evaluation protocols, emphasizing consistency across programs using self-report data. Econoler identified areas for maintenance and updates to align with current best practices.
Aspects Maintained - › The overall adjustment approach to cross-influence currently used in E1 algorithms. This approach captures influences outside the program (e.g. past participation, educational initiatives, etc.), the results of which...
AI summary The text outlines aspects maintained in program evaluation methods, including cross-influence adjustments, spillover capture, consistency checks in NTGR calculations, and questionnaire design. These focus on ensuring accurate free-ridership scoring, evaluating program impacts, and maintaining efficient data collection frameworks.
7 Net-to-gross Ratio (NTGR) Methodology This section documents best practices and recommendations with respect to NTGR calculations, NTGR methodology, algorithm design, free-ridership (FR) and participant spillover (SO) scoring, consistenc...
AI summary This section outlines best practices for calculating the Net-to-Gross Ratio (NTGR), including algorithm design, free-ridership (FR) and participant spillover (SO) scoring, consistency checks, and study design methodologies for regulatory evaluations.
7.1 NTGR Algorithm
AI summary Section 7.1 discusses the NTGR (Net-to-Gross Ratio) algorithm, a method for evaluating demand-side management (DSM) program effectiveness. It focuses on quantifying the ratio of net benefits to gross costs, influencing cost recovery mechanisms and program evaluation frameworks within regulatory proceedings.
Free-ridership Score Components In calculating free-ridership, the self-report method is aimed at establishing how the participant decided to implement the energy efficient or demand response measure(s). Three factors need to be considered...
AI summary The free-ridership score calculation uses a self-report method evaluating three factors: Intention (likelihood of implementing measures without the program), Influence (program's impact on decisions), and Cross-influence (past program efforts' effect on current decisions). These factors assess participant behavior and program effectiveness.
Partial free-rider: - › Program had impact on quantity/timing/efficiency of upgrades, but not full impact - › Aspects of the program influenced their decision to participate
AI summary The program had a partial impact on the quantity, timing, and efficiency of upgrades, though not a full impact. Certain program aspects influenced participants' decisions to engage, indicating partial free-rider effects rather than complete program influence.
Checking for Consistency Ensuring consistency in responses is considered best practice according to the UMP and all jurisdictions studied; this is accomplished by including a series of questions to capture any responses that contradict oth...
AI summary The text emphasizes the importance of consistency checks in free-ridership algorithms, citing the UMP as a best practice. E1's current approach is deemed sufficient, but the Evaluator recommends three enhancements: follow-up questions, response comparison, and open-ended inquiries for BNI customers with large projects.
Table 367: Participant Spillover Types Like Spillover Program-induced actions taken outside of the program that are similar to the actions taken as part of the program. (i.e. if a participant in a program that offers rebates on a menu of m...
AI summary The text discusses 'like' and 'unlike' spillover effects in energy efficiency programs, explaining that like spillover occurs when participants take similar actions outside the program, while unlike spillover involves different actions. Like spillover is more commonly captured in studies, whereas unlike spillover is harder to measure through surveys and often requires educational components in program design.
Spillover Scoring Typically, spillover is captured in a series of three to five questions to discern if any further energy efficiencyrelated actions took place after the participant took part in the program. If the participant took further...
AI summary Spillover scoring assesses additional energy efficiency actions taken by program participants post-enrollment. Savings from these actions are multiplied by a spillover score and divided by total program savings to calculate net spillover impact. Follow-up interviews are recommended for complex measures to estimate savings accurately.
7.2 NTGR Study Design and Sampling The following considerations are important when designing a study to capture the NTGR for any program. › Free-ridership should be explicitly considered for all measures in a program. Where possible, evalu...
AI summary The text outlines key considerations for designing studies to capture the Net-to-Gross Ratio (NTGR) in programs, emphasizing the need to account for free-ridership, ensure robust sampling, and apply weighted NTGR values when multiple measures are involved. Alternative NTGR approaches must be used for excluded measures.
- › The sample size should be designed to aim for a 90% confidence level and a 10% margin of error when based on the available population of projects in a program. Project-level sampling is best practice as survey questions are posed on a...
AI summary The text discusses sampling methodologies for program evaluation, emphasizing project-level sampling for 90% confidence and 10% margin of error. It addresses NTGR scoring adjustments for midstream programs, recommending inclusion of distributor influence on free-ridership, with E1's BER-IR program as a case study. Econoler advises incorporating stakeholder perspectives in NTGR scoring for accurate program evaluation.
Question Design - › Use simple words and avoid technical terms and slang - › Use specific, clear wording rather than general, abstract terms - › Use words that can be interpreted in only one way to avoid ambiguity - › Use question wording...
AI summary The document outlines principles for designing effective survey questions in regulatory proceedings, emphasizing clarity, neutrality, and methodological rigor. It references academic studies and frameworks like the Uniform Methods Project for evaluating energy efficiency programs, billing analysis, and free ridership estimation.
Triangulation As mentioned earlier, triangulation consists of combining one or more data-collection methods. If applied correctly, the combination of methods enables a higher level of precision to be achieved but leads to an increase in ti...
AI summary Triangulation enhances data precision by combining methods but increases complexity, time, and workload. It can involve self-report approaches like interviewing program participants and market actors to compare results and strengthen findings.
8 Analysis of E1's NTGR Approach This section provides an overview of E1's current NTGR approach particularly as it relates to measures included in the NTGR calculation and spillover. An overview is presented on how the NTGR approach shoul...
AI summary This section reviews E1's current NTGR approach, focusing on measures included in the calculation and spillover effects. It evaluates whether the approach should be updated based on best practices, emphasizing the need for alignment with regulatory standards and program effectiveness.
8.1 Review of E1's Current Approach [Table](#page-17-0) 369 below outlines for each program component, the NTGR results, what measures or sub-categories are included in the NTGR, the current type of spillover measured and the last date the...
AI summary This section reviews E1's current approach, focusing on the Net-to-Gross Ratio (NTGR) results for various program components. It outlines the spillover effects measured, the last update dates, and proposed changes for the 2024 Evaluation. Programs targeting low-income participants use a deemed NTGR ratio of 1.
Aspects Maintained - › The overall adjustment approach to cross-influence currently used in E1 algorithms. This approach captures influences outside the program (e.g. past participation, educational initiatives, etc.), the results of which...
AI summary The text outlines maintained aspects of E1 algorithms' cross-influence adjustments, spillover capture methods, consistency checks for NTGR calculations, and questionnaire design for program evaluations. These elements aim to ensure accurate free-ridership measurement and program effectiveness.
9 Program Algorithms In the subsections that follow, two E1 program algorithms are evaluated in more detail – those of EPI and BER-IR as examples as the NTGR for these programs will be updated as part of the 2024 Evaluation. For each algor...
AI summary The document evaluates EPI and BER-IR program algorithms under E1, reviewing alignment with best practices. It outlines current and recommended algorithm elements, including variables, question design, scoring, and consistency checks, with revisions highlighted in light green for the 2024 NTGR update.
Table 373: Instant Rebates Distributor Influence Level Algorithm Current Algorithm Adjusted Algorithm E8 [IF E5=1] Efficiency Nova Scotia promotional materials or communications prompted a company representative to take into account the co...
AI summary This table presents an algorithm related to the influence level of instant rebates distributors, focusing on whether Efficiency Nova Scotia promotional materials prompted company representatives to consider the cost-effectiveness of efficient products. It includes options for agreement, disagreement, or not knowing.
G-Report_FINAL_2021.06.08.pdf)[PRODNTG\_Res-Products-NTG-Report\_FINAL\_2021.06.08.pdf.](https://www.ma-eeac.org/wp-content/uploads/MA20X04-E-PRODNTG_Res-Products-NTG-Report_FINAL_2021.06.08.pdf) NMR Group, Inc and DNV for Massachusetts Pr...
AI summary The document references multiple reports and memoranda related to energy efficiency program methodologies, focusing on Net-to-Gross Ratios (NTGR) for fuel-switching heat pumps and residential self-report measurement updates. Key entities include NMR Group, DNV, Tetra Tech, and Massachusetts Program Administrators, emphasizing standardized evaluation approaches for energy savings.