Topic:"Free Ridership Measurement" in M12186

Table 1 Update on Implementation of 2018-2022 Evaluation Recommendations Year Evaluation/ Verification Recommendation Text Source Status Comments Expected Period of Completion 2022 Evaluation Monitor free-ridership for this service and und...

AI summary In 2022, an evaluation recommended monitoring free-ridership and conducting a process evaluation for New Construction by 2023 or 2024. EfficiencyOne (E1) agreed, and a comprehensive impact evaluation was conducted in 2023. A process evaluation was also conducted in 2024, with results expected to be filed with the Nova Scotia Utility and Review Board in March 2025.

Surveys and Interviews This subsection describes the data-collection activities conducted for the impact evaluations. It should be noted that surveys and interviews were often integrated to collect impact, process, and market information a...

AI summary This subsection outlines the data collection methods used in impact evaluations, including participant and non-participant surveys conducted between September and December 2024. Surveys were primarily conducted by telephone, with exceptions for online formats used in Instant Savings, Green Heat, and Home Energy Assessment programs.

Site Visits and Project Reviews with Follow-up Site Visits or Interviews The Evaluator performed a total of 146 project reviews during the summer and fall of 2024, 70 of which were complemented through site visits and 14 were complemented...

AI summary The Evaluator conducted 146 project reviews in 2024, including site visits and phone interviews, to validate installations and gather information on free-ridership and spillover effects for various programs such as Efficient Product Installation, Affordable Multifamily Housing, and Strategic Energy Management.

Net-to-gross Assessment and Net Savings Calculations Free-ridership levels were established for select program components by conducting self-report surveys or in-depth interviews. Those surveys and interviews included a set of questions us...

AI summary The document outlines methods for calculating free-ridership and spillover levels in energy efficiency programs, using surveys and interviews. It details updates for 2024 evaluations, excludes low-income participants due to nil free-ridership, and references NTGR reviews and spillover assessments for various program components.

Business Energy Rebates - › In 2024, Instant Rebates participation decreased by 26% compared to 2023 levels. Application Rebates participation increased by 18% in 2024, while gross electrical energy savings per Application Rebates particip...

AI summary In 2024, Business Energy Rebates (BER) saw a 26% drop in Instant Rebates participation but an 18% rise in Application Rebates. Free-ridership for LED products decreased after algorithm updates, while NTGR improvements increased energy savings. Evaluated savings for Application Rebates matched E1's data, but Instant Rebates showed 4-6% higher savings.

Custom - › Compared to 2023, Custom participation decreased in 2024. While Retrofit and Building Optimization participation levels decreased, New Construction participation significantly increased in 2024. - › Following the project reviews...

AI summary Custom program participation decreased in 2024, with Retrofit and Building Optimization participation declining while New Construction increased. The Evaluator adjusted energy and peak demand savings estimates, noting reduced free-ridership (15% for Retrofit, 28% for New Construction) and no spillover. Evaluated savings were 8% and 5% higher than E1's tracked figures.

3.2 Net-to-gross Ratios The NTGRs outlined in [Table](#page-30-1) 7 below were applied to gross savings to estimate net savings. NTGRs were established based on the free-ridership levels and spillover in some cases. Based on the establishe...

AI summary The document discusses the calculation of Net-to-gross Ratios (NTGRs) used to estimate net savings from energy efficiency programs. NTGRs are determined based on free-ridership and spillover levels, using the equation NTGR = (1 - % Free-ridership + % Spillover). The methodology relies on self-reporting and was described in a prior subsection.

Table 7: 2024 Free-ridership, Spillover, and NTGRs Program Component and Measure Type Free-ridership Levels Spillover Levels NTGR Residential Appliance Retirement a Refrigerators 43% 0% 0.57 Freezers 45% 0.55 Air Conditioners 47% 0.53 Smal...

AI summary Table 7 presents data on free-ridership, spillover, and net-to-gross ratios (NTGR) for various energy efficiency programs in 2024. It includes metrics for residential and non-residential programs, highlighting levels of free-ridership and spillover across different measures and participant categories.

This appendix presents some examples of how the Evaluator established the margins of error for participant surveys, adjustment ratios, installation rates, and free-ridership levels. For participant survey margins of error, the example is d...

AI summary This appendix provides examples of how the Evaluator determined margins of error for various metrics, including participant surveys, adjustment ratios, installation rates, and free-ridership levels, using evaluations from 2024 Green Heat, 2024 Custom Retrofit, and the Efficient Product Installation (EPI) evaluation.

Free-ridership Level Margin of Error The Evaluator used the margin of error calculation of the free-ridership level for EPI LED lighting in 2024 as an example. Below are the steps followed to calculate this margin of error. The same calcul...

AI summary The Evaluator demonstrated the margin of error calculation for EPI LED lighting's free-ridership level in 2024, noting the same method applies to installation rates and adjustment ratios for non-stratified samples by substituting relevant variables.

Calculation of the Weighted Average of Free-ridership Levels The weighted average was calculated by using the following formula. $$Weighted\ Average\ FR = \frac{\sum_{i=1}^{n} Free\text{-}ridership\ Level}_{i} \times Revised\ Gross\ Energy...

AI summary The weighted average free-ridership level was calculated as 16% using data from participants who installed LED lighting in 2024. The formula involved summing free-ridership levels multiplied by revised gross energy savings, divided by total revised gross energy savings.

Calculation of the Standard Error Since the free-ridership level is based on a weighted average, the Evaluator also calculated a weighted standard error of the free-ridership level instead of a simple standard error. The same formula as fo...

AI summary The Evaluator calculated a weighted standard error for the free-ridership level using a formula similar to that for adjustment ratios, resulting in a value of 0.0193. The calculation involved the sample size, participant-specific free-ridership savings, and average revised gross savings per participant.

Calculation of the Margin of Error The margin of error on the adjustment ratio of the EPI LED lighting free-ridership level was established by using the following formula that is the general equation linking the standard error to the margi...

AI summary The margin of error for EPI LED lighting free-ridership was calculated using a formula involving standard error (0.0193), a t-value (1.667 for 90% confidence), and a sample size (n=70 from N=5,342 participants). The result was 3.2%, applied to EPI, Instant Savings, BER Instant Rebates, Custom programs, EPI installation rates, and AMH adjustment ratios in 2024.

APPENDIX III NTGR Calculations This appendix provides an example of net-to-gross ratio (NTGR) calculations. The example details the calculations of participant free-ridership levels and resulting NTGR for EPI LED Bulbs. The Evaluator used...

AI summary This appendix details an example of net-to-gross ratio (NTGR) calculations for EPI LED Bulbs, including participant free-ridership level assessments. The Evaluator applied a similar methodology to other program components, with further details available in individual program evaluation reports.

NTGR Calculations Free-ridership algorithm High Free-ridership Participant Medium Free-ridership Participant Questions Calculation Algorithm Answer Calc. Answer Calc. Answer Calc.

AI summary The document outlines a table related to free-ridership algorithms, focusing on high and medium free-ridership participants. It includes columns for questions, calculation algorithms, answers, and calculations, but no specific details or context are provided in the text.

Instant Savings Findings and Recommendations This subsection presents the key findings from 2024 Instant Savings evaluation. The Evaluator has no specific recommendation for Instant Savings. 2024 Instant Savings-Finding: Instant Savings su...

AI summary 2024 Instant Savings exceeded energy savings (77%) and peak demand savings (45%) targets. Participation rose 82% due to E1's campaign, with LED lighting driving most savings. Non-lighting savings increased, free-ridership dropped, and evaluator-estimated savings outpaced E1's tracked results.

3.3.1 Free-ridership and Secondary Market Impacts For appliance retirements, free-ridership corresponds to the energy consumption of appliances that would have been disposed of in the absence of the program, and secondary market impacts co...

AI summary The text discusses free-ridership and secondary market impacts in the context of appliance retirement programs. It notes that data from 2024 was unavailable, so 2022 and 2018 data were used for calculations. Free-ridership refers to energy consumption from appliances that would have been disposed of without the program, while secondary market impacts refer to appliances transferred to other owners.

Table 11: 2024 ARet Free-ridership Levels Appliance Type Free-ridership (with Secondary Market Impacts) Level Margin of Error Refrigerators 43% 6.3% Freezers 45% 5.5% Air Conditioners 47% 13.0% Small Refrigerators 32% 10.3% Small Freezers...

AI summary Table 11 presents 2024 ARet free-ridership levels for various appliance types, indicating the percentage of participants who may not have been directly targeted by the program but still benefited from it. The data shows varying levels of free-ridership across different appliance categories, with refrigerators at 43% and dehumidifiers at 0%. The section 'Participant Spillover' discusses the implications of these findings.

3.3.3 Net-to-gross Ratio Calculation NTGR values were calculated using the following equation. $$NTGR = (1 - \% Free-ridership + \% Spillover)$$ Using the average free-ridership levels established for each appliance type, the Evaluator cal...

AI summary The document discusses the calculation of the Net-to-gross Ratio (NTGR) using the formula NTGR = (1 - % Free-ridership + % Spillover), with results based on average free-ridership levels for various appliance types.

Table 12: 2024 ARet Effects and NTGR Appliance Type Average Free-ridership (with Secondary Market Impacts) Spillover NTGR Refrigerators 43% 0% 0.57 Freezers 45% 0.55 Air Conditioners 47% 0.53 Small Refrigerators 32% 0.68 Small Freezers 32%...

AI summary Table 12 presents the 2024 ARet Effects and NTGR for various appliance types, including free-ridership rates and spillover impacts. The table highlights the percentage of free-ridership for refrigerators, freezers, air conditioners, small refrigerators, small freezers, and dehumidifiers, along with NTGR values for each appliance type.

6 Instant Savings Evaluation Approach The 2024 Instant Savings evaluation included a comprehensive impact evaluation whereby NTGRs, namely free-ridership, as well as unitary savings were reviewed and updated. The main objectives of the 202...

AI summary The 2024 Instant Savings evaluation aimed to calculate gross and net savings, including energy and peak demand savings, and avoided GHG emissions. The evaluation reviewed free-ridership and unitary savings, with research questions and methods outlined in a table.

Table 18: 2024 Instant Savings Evaluation Approach Evaluation Objective Research Question Methodology Calculate gross results › Are the data in the tracking sheet complete, accurate, and consistent? › What are the unitary savings and EUL v...

AI summary This section outlines the methodology for evaluating the 2024 Instant Savings program, focusing on calculating both gross and net results. It includes an audit of tracking sheets, unitary savings review, EUL updates, participant surveys, and calculations for energy savings and GHG emission reductions.

Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...

AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations, focusing on free-ridership levels in the Instant Savings program. Confidence levels exclude non-sampling errors. The 2024-2025 DSM MA document provides parameters for calculating energy savings and effective useful life values for E1's DSM program measures.

7.3.1 Free-ridership Free-ridership levels are calculated using algorithms that assess the likelihood of participants purchasing LED lamps or fixtures without the discount offered through Instant Savings. The algorithms consider all applic...

AI summary Free-ridership levels for LED purchases under Instant Savings are calculated using algorithms and surveys, with distinct analyses for Costco and non-Costco participants. Data from 2023 and 2024 were used, noting insufficient 2024 responses for non-Costco fixture buyers. Nova Scotia Power and Emera Inc. provided emissions and generation data for calculations.

Measurement of Free-ridership for Costco Participants The free-ridership results for Costco participants are presented in [Table](#page-121-0) 25 below.

AI summary The document presents free-ridership results for Costco participants in Table 25. This analysis is relevant to understanding the extent to which Costco participants may be benefiting from energy efficiency programs without contributing proportionally to their costs.

Table 25: 2024 Instant Savings Free-ridership for Costco Participants Free-ridership Product Category Level Margin of Error LED Lamps 16% 7.1% LED Fixtures 19% 9.1% Recessed Fixtures 18% 4.7% For Costco participants, the intercept survey r...

AI summary The free-ridership level for Costco participants in the 2024 Instant Savings program was 16% for LED lamps, a decrease from 36% in 2022. This is attributed to fewer Costco participants being unaware of the discount and a lower proportion of people purchasing products at full price or the same quantity without the discount.

Measurement of Free-ridership for Non-Costco Participants – LED Lamps The free-ridership results for LED lamps for non-Costco participants are presented in [Table](#page-121-1) 26 below.

AI summary The document presents free-ridership results for LED lamps for non-Costco participants in Table 26. This analysis is part of a regulatory proceeding in Nova Scotia.

Table 26: 2024 Instant Savings Free-ridership for Non-Costco Participants Free-ridership Product Category Level Margin of Error LED Lamps 51% 14.7% For non-Costco participants, the intercept survey revealed a free-ridership level of 51% fo...

AI summary The 2024 Instant Savings Free-ridership for Non-Costco Participants shows a free-ridership level of 51% for LED lamps, lower than the 70% in 2023. Despite a margin of error over 10%, the Evaluator used this result as Costco also showed a similar reduction.

Treatment of Free-ridership for Non-Costco Participants – LED Fixtures The 2023 results for non-Costco participants that purchased LED fixtures were based on an online survey that generally used the same questions and algorithms for LED la...

AI summary The 2023 results for non-Costco participants who purchased LED fixtures were determined using an online survey with similar questions and algorithms as in 2024, and the results are presented in Table 27.

Table 27: 2024 Instant Savings Free-ridership Levels for Non-Costco Participants Draduct Catagory Fre ee-Ridership Product Category Level Margin of Error All LED Fixtures 48% 10.3% 2024 Overall Free-ridership Levels

AI summary The document presents Table 27, which outlines the 2024 free-ridership levels for non-Costco participants under the 'All LED Fixtures' product category, indicating a 48% free-ridership level with a margin of error of 10.3%.

The overall free-ridership levels for LED lamps and fixtures correspond to the average of the 2024 Costco free-ridership levels, the 2024 non-Costco free-ridership levels for LED lamps, and the free-ridership of non-Costco consumers for LE...

AI summary The free-ridership levels for LED lamps and fixtures are calculated as an average of Costco and non-Costco free-ridership levels, weighted by the number of units sold in 2024. Table 28 provides the detailed free-ridership levels for these products.

Table 28: 2024 Instant Savings Free-Ridership for LED Lamps and Fixtures Product Retailer Evaluation Free-ridership Number of Units Sold ited Average e-ridership Category Period Level Margin of Error in 2024 Level Margin of Error LEDiama C...

AI summary Table 28 presents free-ridership levels for LED lamps and fixtures sold through Instant Savings in 2024. Free-ridership refers to the percentage of savings that are not captured by the program. The data shows varying levels of free-ridership depending on the product and retailer. Table 29 summarizes free-ridership for all product categories, indicating that all other products had zero free-ridership.

The NTGR is calculated using the following equation. $$NTGR = (1 - \% Free-ridership + \% Spillover)$$ Using this equation, the NTGRs for LED lamps and LED fixtures were estimated at 0.70 and 0.76 respectively, as presented in [Table](#pag...

AI summary The NTGR is calculated using the equation (1 - % Free-ridership + % Spillover). For LED lamps and fixtures, NTGRs were estimated at 0.70 and 0.76, respectively. For other products, NTGR was set at 1.00 due to challenges in assessing free-ridership and spillover levels, particularly for smaller population segments and specialized products.

8 Instant Savings Key Findings and Recommendations The 2024 Instant Savings evaluation included a comprehensive impact evaluation whereby NTGRs, notably for free-ridership, as well as unitary savings were reviewed and updated. The main obj...

AI summary The 2024 Instant Savings program exceeded energy and peak demand savings targets by 77% and 45%, driven by E1's campaign and LED product adoption. Participation rose 82% due to rebate deadlines, with LED lighting accounting for 69% of savings. Free-ridership dropped to 39% for LED lamps and 26% for fixtures. Evaluated savings were 16% higher than E1's tracked figures due to updated NTGRs.

Instant Savings Appendix III Instant Savings: Participant Online Survey Questionnaire Appendix IV Instant Savings: Participant Survey Results Appendix V Instant Savings: Tracking Sheet Audit Appendix VI Instant Savings: Algorithms for Free...

AI summary The document outlines appendices related to the 'Instant Savings' program, including participant surveys, audit tracking sheets, free-ridership calculation algorithms, and 2024 recommendations. It also includes a project number and contact details for a Quebec-based organization involved in the proceeding.

Table 1: Overview of Data Collection Activity Descriptor This Instrument Instrument Type Participant Intercept Survey with Online Option Estimated Time to Complete 5 min Target Audience Instant Savings participants who have purchased LED n...

AI summary This document outlines the data collection activities for a survey targeting participants of the Instant Savings program who purchased LED bulbs and fixtures during the fall 2024 campaign. The survey aims to assess free-ridership, cross-influence, and participant perspectives on program awareness.

G. Free-Ridership – LED Fixtures - G1. Efficiency Nova Scotia offered a discount on LED fixtures. Before paying at the cash register, were you aware that a discount was offered on the purchase of LED fixtures? [ALLOW ONLY ONE CODE] - 1. Ye...

AI summary The document presents a series of questions assessing awareness of Efficiency Nova Scotia's LED fixture discount and its influence on purchasing decisions, aiming to evaluate free-ridership in the program. Respondents are asked about prior knowledge of the discount and motivations for purchasing LED fixtures.

D. Free-Ridership - LED Bulbs - D1. Efficiency Nova Scotia offered a discount on non-pear-shaped LED bulbs. Before paying at the cash register, were you aware that a discount was offered on the purchase of LEDs?

AI summary This section addresses free-ridership related to LED bulbs, specifically asking if customers were aware of a discount offered by Efficiency Nova Scotia on non-pear-shaped LED bulbs before checkout.

F3. The day you recently bought the LED light fixture(s), did you see signs or posters in the light fixture section of the store promoting the discount offered on LED fixtures? 2024 2023 2022 2021 Sample Size 159 41 128 53 Yes 72% 76% 37%...

AI summary This section asks about the visibility of promotional signs for LED light fixture discounts in the store. A table shows response rates across different years, and a section titled 'Free-Ridership – LED Fixtures' is introduced.

Table 1: FR Algorithm - LED Bulbs Section 1 – Free-ridership level for those who were unaware of discount before paying D1. Before paying at the cash register, were you aware that a discount was offered on the purchase of LEDs? IF Yes : GO...

AI summary The document presents a table outlining a free-ridership algorithm for LED bulbs, focusing on individuals unaware of discounts before purchasing and those who postponed purchases due to the program. It includes sections with questions and decision points to determine free-ridership levels.

Table 2: ME Algorithm - LED bulbs Total Non-A Type Sales Non-A type LED bulbs sold from January – December 2024(extrapolated from retailer data and responses in interview) #LEDJan-Dec LED Sales During Instant Savings' Campaigns LED Bulb Sa...

AI summary The text presents a table detailing the calculation of final market effects for LED bulb sales, including the influence of Efficiency Nova Scotia's programs and the determination of free-ridership levels based on customer awareness of discounts.

DEFINITIONS Accuracy Reflects the proximity of measurements to the true value. Adjustment ratio The ratio of evaluated results to tracked results. This ratio expresses the adjustment made to tracked savings or other tracked values such as...

AI summary The text defines key terms related to demand response (DR) and energy efficiency, including available DR capacity, adjustment ratio, and baseline. It also references a free-ridership section (14.3.1) in the document.

4.3 Net Savings The Evaluator determined the net energy and peak demand savings, i.e. the electrical energy and peak demand savings that can be reliably attributed to a program component, by estimating the net-to-gross ratio (NTGR). More p...

AI summary The Evaluator calculated net energy and peak demand savings using the net-to-gross ratio (NTGR), accounting for spillover and free-ridership effects. For AMH, NTGR was set to 1.00 due to negligible free-ridership and spillover impacts from participants' limited budgets for building improvements.

4.3.1 Evaluated Net Savings Net savings are defined as the changes in energy use that are specifically attributable to AMH. Since spillover and free-ridership effects were considered nil, the net energy savings are equal to the gross savin...

AI summary The section defines net savings for AMH (Affordable Multifamily Housing) as gross savings, assuming no spillover or free-ridership effects. However, AMH missed its 2024 electrical energy and peak demand savings targets by 39% and 64%, respectively, as shown in Figure 14.

9.3 Net Savings The Evaluator determined the net energy and peak demand savings, i.e. the electrical energy and peak demand savings that can be reliably attributed to a program component, by estimating the net-to-gross ratio (NTGR). More p...

AI summary The Evaluator calculated net energy and peak demand savings using the net-to-gross ratio (NTGR), which accounts for spillover and free-ridership effects. For Affordable Single-family Homes (ASFH), these effects were considered nil due to participants being non-profits or low-income housing owners, leading to a NTGR of 1.00.

Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were condu...

AI summary The Evaluator aimed for a 10% margin of error at 90% confidence level in evaluating installation rates, free-ridership, and spillover levels. Only random sampling errors were considered, excluding non-sampling errors like data entry issues or response inaccuracies.

14.3 Net Savings The Evaluator determined the net energy and peak demand savings, i.e. the electrical energy and peak demand savings that can be reliably attributed to a program component, by estimating the NTGR. In the case of EPI, the NT...

AI summary The Evaluator calculated net energy and peak demand savings by estimating the NTGR, accounting for free-ridership and participant spillover effects in EPI programs. This approach adjusts for unintended influences on energy savings outcomes.

14.3.1 Free-ridership For EPI, free-ridership occurs when participants would have installed the same energy efficient products in the absence of the program component. For low-income participants, the free-ridership level is assumed to be...

AI summary The document discusses free-ridership in EPI, noting that non-low-income participants were surveyed to assess program impact, while literature reviews informed air sealing product free-ridership levels. A 12% rate was selected for air sealing based on Massachusetts data, reflecting lower adoption likelihood without the program.

Table 35: 2024 EPI Participant Free-ridership Levels Average Free-ridership Level Margin of Error Low-income Participants All Products 0% N/A Non-low-income Participants LED Lighting and Lighting Related Products 16% 3.2% Smart and Advance...

AI summary Table 35 presents free-ridership levels for EPI participants in 2024, showing a reduction in LED lighting free-ridership from 25% in 2021 to 16% in 2024. This decrease is attributed to an updated algorithm that increased the influence weight of EPI and E1, resulting in lower free-ridership.

14.3.2 Participant Spillover For EPI, participant spillover occurs when participants purchase and install additional energy efficient products due to the influence of having participated in the program component without receiving any addit...

AI summary The document discusses participant spillover in the EPI program, where participants install additional energy-efficient products post-program participation without additional support. The 2024 evaluation identified LED lamps, fixtures, and heat pump water heaters as the main products driving spillover, with 17 out of 100 surveyed participants reporting such behavior.

19.3.1 Free-ridership For Green Heat, free-ridership occurs when participants would have still installed a new, more efficient heating system in the absence of the program component. Free-ridership was assessed during the 2023 evaluation u...

AI summary Free-ridership in the Green Heat program occurs when participants would have installed more efficient heating systems without the program. In 2023, free-ridership was assessed using self-reporting, and the same levels were used for 2024 due to no new data collection.

Table 48: 2024 Green Heat Average Free-ridership Levels Measure Average Free-ridership Level Sample Size Population Size Margin of Error MSHPs 48% 90 1,132 5.5% Biomass Measures 47% 29 213 8.2% Demand Reduction Measures 9% 5 314 13.4% CASH...

AI summary Table 48 presents the 2024 average free-ridership levels for various energy efficiency measures under the Green Heat program, including mini-split heat pumps, biomass measures, demand reduction measures, and CASHPs and AWHPs. The data includes sample sizes, population sizes, and margin of error for each measure.

19.3.2 Net-to-gross Ratio Calculation The NTGR was calculated using the following equation. $$NTGR = (1 - \% Free-ridership)$$ Using the free-ridership level established for each measure category, the Evaluator calculated the NTGR values p...

AI summary The Net-to-gross Ratio (NTGR) is calculated as (1 - % Free-ridership), with the Evaluator using established free-ridership levels for each measure category to determine NTGR values presented in Table 49.

23.3.1 Free-ridership For HEA, free-ridership occurs when participants would have implemented energy efficiency measures and/or solar PV systems in the absence of the program component. Free-ridership was assessed in 2023 through a partici...

AI summary Free-ridership in the Home Energy Assessment (HEA) program occurs when participants would have implemented energy efficiency measures or solar PV systems without the program. Free-ridership levels from 2023 were used for the 2024 evaluation since no data collection was conducted in 2024.

23.3.2 Participant Spillover For HEA, participant spillover occurs when participants implement additional measures recommended in their initial energy assessments after their participation in the program component, that is after having com...

AI summary Participant spillover in the Home Energy Assessment (HEA) program occurs when participants implement additional energy efficiency measures after completing their initial assessments, without receiving further support from the program. The 2023 spillover level was reused in the 2024 evaluation due to a lack of updated data collection.

The NTGR is calculated using the following equation. NTGR = (1 – % Free-ridership + % Participant Spillover) The NTGR values calculated with the free-ridership and spillover levels established for HEA are presented in [Table](#page-165-1)...

AI summary The NTGR is calculated using the formula (1 – % Free-ridership + % Participant Spillover), with values based on free-ridership and spillover levels established for HEA, as presented in Table 64.

Efficient Product Installation Appendix XIII: EPI Tracking Sheet Audit Appendix XIV: EPI Participant Survey Questionnaire Appendix XV: EPI Participant Survey Results Appendix XVI: EPI On-site Visit Sampling Methodology and Protocol Appendi...

AI summary The document outlines appendices related to Efficient Product Installation (EPI), including audit processes, participant surveys, free-ridership/spillover calculation algorithms, jurisdictional findings, and 2024 recommendations. These materials support program evaluation and implementation methodologies.

E. Free-Ridership @INSTRUCTION: FREE-RIDERSHIP SERIES SHOULD BE ASKED ONLY TO RESPONDENTS WHO INDICATED: "YES" IN [A1](#page-101-0)[A](#page-101-1) (LED BULBS) – ASK FR LED BULBS SECTION AND/OR "YES" IN A1M (SMART THERMOSTAT) – ASK FR SMAR...

AI summary The text outlines conditions for asking free-ridership questions in a regulatory proceeding, targeting respondents who indicated 'YES' to LED bulb or smart thermostat programs. Respondents are directed to specific sections based on their affirmative answers.

Lighting Products Product Quantity in Tracking Sheet Quantity On Site Explanation, If Different Indoors or Outdoors 9 W LED 18 W LED 7 W PAR20 LED 15 W PAR38 LED 7 W GU10 LED 7 W G25 LED Globe 5 W E12 LED Chandelier LED Nightlight Notes: D...

AI summary The document contains tables detailing various lighting and domestic hot water (DHW) products, including quantities in tracking sheets and on site, along with notes and explanations. It also includes a section on thermostats and power strips. An appendix discusses EPI algorithms for free-ridership calculation.

Table 1: Free-Ridership - LEDs Previous Algorithm n 2024 AI gorithm Question Answer Score Question Answer Score INTE NTION Planr ning Had you already decided to purchase 1) Yes Use E2 Before learning about the Efficient 1) Yes Use E2 E1 an...

AI summary This table presents a comparison of two algorithms used to assess free-ridership related to LED bulb installations. It includes questions and responses from participants, such as whether they had already decided to purchase LED bulbs before learning about the Efficient Product Installation Service, along with scores and adjustments based on their answers.

Table 3: Free-Ridership – Smart Thermostats Previous Algorithm 2024 Algorithm Question Answer Score Question Answer Score INTENTION Planning Before learning about the Efficient Product 1) Yes Use E8 E7 Installation Service, had you already...

AI summary Table 3 outlines a free-ridership analysis related to smart thermostats, comparing the previous algorithm with the 2024 algorithm. It includes questions about customer intent and cost considerations, such as whether customers had already decided to install smart thermostats before learning about the Efficient Product Installation Service and the likelihood of purchasing them without the service.

Table 5: Free-Ridership – Domestic Hot Water Measures Previous Algo orithm 2024 AI gorithm Question Answer Score Question Answer Score INTE NTION Planr ning Had you already decided to 1) Yes Use E8 5 ( ) 1) Yes Use E14 E13 purchase and ins...

AI summary Table 5 discusses free-ridership related to domestic hot water measures, including questions about prior decisions to install low flow showerheads and other efficiency measures before learning about the Efficient Product Installation Service. The table includes scoring and response options.

Table 5: 2024 BER Evaluation Approach Evaluation Objectives Research Questions Methodology Collect information on participant and distributor perspectives › How do participants become aware of BER-IR? › What is the level of satisfaction wi...

AI summary Table 5 outlines the 2024 BER Evaluation Approach, focusing on collecting participant and distributor perspectives, calculating gross and net results, and using methods such as surveys, interviews, and adjustment ratios. It also references the 2024-2025 DSM MA for evaluating energy savings and GHG emission reductions.

Note on Margin of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were conduc...

AI summary The Evaluator aimed for a 10% margin of error at 90% confidence for quantitative evaluations. BER-AR evaluations did not require margin calculations, while BER-IR included free-ridership margin calculations detailed in Appendix II of the 2024 DSM Programs Evaluation Executive Summary.

4.3.1 Free-ridership In the case of Application Rebates, free-ridership occurs when participants would have still implemented energy efficiency measures in the absence of the service. The free-ridership level was assessed during the 2021 e...

AI summary Free-ridership in Application Rebates refers to participants who would have implemented energy efficiency measures without the rebate. The 2021 evaluation used a self-report approach via telephone survey, and this level was carried forward for the 2024 evaluation as no new data collection occurred.

Table 12: 2024 Application Rebates Free-ridership Measure Category Average Free-ridership Level All 26% 6.6% 4.3.2 Spillover

AI summary Table 12 presents free-ridership levels for the 2024 Application Rebates, with an average free-ridership level of 26% and 6.6% for all measures. Section 4.3.2 discusses spillover effects related to these rebates.

Table 13: 2024 Application Rebates NTGR Measure Category Free-ridership Spillover NTGR All 26% 0% 0.74 4.3.4 Evaluated Net Savings

AI summary Table 13 provides information on the 2024 Application Rebates NTGR, including free-ridership and spillover percentages. Section 4.3.4 discusses the evaluated net savings, highlighting the need for consideration of these factors in rebate programs.

5.3.1 Free-ridership For Instant Rebates, free-ridership corresponds to the proportion of savings attributed to natural market trends. Thus, the free-ridership assessment is aimed at estimating the level of sales of energy efficient produc...

AI summary This section discusses the assessment of free-ridership for the Instant Rebates program, focusing on the proportion of energy savings attributed to natural market trends rather than program influence. The assessment uses self-report data from participants and distributor interviews, with updated algorithms applied in 2024. The analysis covers LED lighting products, which account for 95% of energy savings from the program.

Table 22: 2024 Instant Rebates Free-ridership Measure Category Average Distributor Influence Level (from distributor interviews) Free-ridership Level (from participant survey) Overall Free-ridership Level Margin of Error LED Linear Fixture...

AI summary Table 22 presents free-ridership levels for various LED lighting measure categories under the 2024 Instant Rebates program. The data shows that free-ridership levels range from 8% to 15%, with the lowest being for LED linear fixtures and the highest for LED outdoor fixtures. The table also includes influence levels from distributor interviews and participant surveys.

5.3.2 Net-to-gross Ratio The NTGR is calculated using the following equation. $$NTGR = (1 - \% Free-ridership)$$ As presented in [Table](#page-68-1) 23, the NTGRs for the main measure categories were established using the freeridership lev...

AI summary The document discusses the Net-to-gross Ratio (NTGR), which is calculated as (1 - % Free-ridership), and explains how it was applied to different measure categories based on free-ridership levels. A NTGR of 1.00 was used for measures sold through Instant Rebates. It also mentions that 'low' influence and intention scores correspond to low free-ridership levels.

in Instant Rebates savings generated by LED linear lamps and LED linear fixtures. 2024 BER Finding: Overall satisfaction with BER Instant Rebates is high among both participants and distributors. For Instant Rebates, average satisfaction r...

AI summary The 2024 BER evaluation highlights high satisfaction (9.1/10 by participants, 8.3/10 by distributors) with Instant Rebates, an updated free-ridership algorithm showing 8-15% free-ridership, and 4-6% higher savings in Application Rebates compared to Instant Rebates. The algorithm now balances intention and influence scores, contributing to lower free-ridership due to increased 2023-2024 incentives.

Business Energy Rebates Appendix I BER: Instant Rebates Participant Survey Questionnaire Appendix II BER: Instant Rebates Participant Survey Results Appendix III BER: Instant Rebates Interview Guide with Distributors Appendix IV BER: Appli...

AI summary The document outlines appendices for Nova Scotia's Business Energy Rebates (BER) program, including surveys, audit tracking sheets, free-ridership algorithms, and 2024 recommendations. EfficiencyOne is identified as the organization involved in the program's implementation.

D. Free-Ridership - D1. [ASK ALL] Were you aware that you received a rebate when purchasing these ? [SINGLE RESPONSE] - 1. Yes - 2. No - 98. (Don't know) - D2. [ASK IF [D1=](#page-95-0)2 OR 98] Have I understood correctly that you were not...

AI summary The text outlines survey questions to assess awareness of rebates for efficient products, channels of awareness, and purchase decisions influenced by rebate availability. It focuses on Efficiency Nova Scotia's Business Energy Rebates Program and evaluates potential free-ridership by asking respondents about pre-rebate purchase intentions.

APPENDIX VI BER Instant Rebates Algorithm for Free-Ridership Calculations Table 1 and Table 2 below present the algorithm[s](#page-142-1) 1 used to calculate the free-ridership levels for Instant Rebates measures. The algorithms are based...

AI summary The appendix outlines algorithms for calculating free-ridership levels in BER Instant Rebates, using participant surveys and distributor interviews to assess program influence on decision-making. The 2024 evaluation updated algorithms following a Net-to-gross Review.

Table 2: Instant Rebates Participant and Overall Free-ridership Algorithm 2022 Algorithm 2024 J Algorithm Influence Score (IS) (10–D12) x 10% INFLUENCE Score (IS) D12 Participant Free-Ridership (FR): MIN (MEAN (CS; PS; IS); MEAN (CS; PS; I...

AI summary Table 2 outlines the algorithm used to calculate participant and overall free-ridership for the Instant Rebates program, comparing the 2022 and 2024 algorithms. The 2022 algorithm uses a mean of multiple scores, while the 2024 algorithm incorporates an intention score and influence score in its calculation.

DEFINITIONS Accuracy Reflects the proximity of measurements to the true value. 4.2 Aw areness and Motivations .27 4.3 Ва rriers to High-Efficiency Buildings and Program Participation .28 4.4 Pa rticipant Satisfaction .29 4.4 .1 Challenges...

AI summary The text defines key terms and outlines sections of an evaluation report, covering topics such as accuracy, participant motivations, barriers to high-efficiency buildings, market factors, decarbonization, and program logic models. It discusses the evaluation of new construction impact, gross savings, net savings, and free-ridership.

Custom General Key Findings and Recommendations 2024 Custom-Finding: Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in net peak demand savings at the generator in 2024, thereby surpassing the planned electrical en...

AI summary Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in peak demand savings in 2024, exceeding targets by 48% and 28%, respectively. Retrofit and Building-Optimization participation declined, while New Construction increased. Adjustments to savings included ratios of 1.012–1.068 across services. Free-ridership dropped to 15% for Retrofit and 28% for New Construction. Evaluated savings were 7% higher than E1's tracked data.

Table 5: Implementation Status of Past Recommendations for Custom # Recommendations Status Comments 2022 Retrofit R2 Require measurement and verification (M&V) efforts based on the International Performance Measurement and Verification Pro...

AI summary The document discusses the implementation status of past recommendations related to custom programs, including the use of M&V efforts based on the IPMVP Option C approach for retrofit projects, revisions to model review processes, monitoring free-ridership, and updating logic models for Custom Retrofit. Progress and completion statuses are outlined with specific comments.

Project File Reviews and Participant Site Visits or Follow-up Phone Interviews In the fall of 2024 and January 2025, Econoler and its subcontractor CDM Energy Solutions carried out a full technical review of project documentation for 19 re...

AI summary In fall 2024 and January 2025, Econoler and subcontractors conducted technical reviews of 19 Retrofit, 1 compressed air leak Retrofit, 2 Building Optimization, and 12 New Construction projects. Site visits and phone interviews were used to finalize reviews and collect free-ridership data for Retrofit projects, while New Construction reviews relied solely on project files. Appendices outlined protocols and interview guides for these processes.

3.3.1 Free-ridership In the case of Retrofit, free-ridership occurs when participants would have still implemented energy efficiency upgrades and measures in the absence of the service. For compressed air leak audit projects, the NTGR was...

AI summary The text discusses the assessment of free-ridership in energy efficiency programs, specifically Retrofit and compressed air leak audit projects. It notes that free-ridership levels for 2024 were estimated based on previous data and self-report methods. The evaluation used phone interviews and an algorithm to calculate free-ridership levels, which are presented in a table with corresponding margin of error.

Table 14: 2024 Retrofit Free-ridership Level per Project Category Retrofit Project Category Free-ridership Level Margin of Error Regular Retrofit 15% 2.90% Solar PV 38% 9.5% Compressed Air Leak Audit 16% 10.6% \ Free-ridership for solar PV...

AI summary Table 14 presents the 2024 free-ridership levels for different retrofit project categories, including Regular Retrofit, Solar PV, and Compressed Air Leak Audit, along with their respective margins of error. The free-ridership level for Solar PV is based on the 2023 evaluation, as it was not measured in 2024.

5.3 Net Savings The NTGR is applied to calculate net savings, that is, the savings that can be reliably attributed to a service. For New Construction, the NTGR was established by considering free-ridership. Spillover was assumed to be zero...

AI summary The NTGR is used to calculate net savings by accounting for free-ridership. For New Construction, spillover was assumed zero due to low nonparticipant spillover, as shown by the 2022 evaluation, leading to no measurement of spillover.

5.3.1 Free-Ridership In the case of New Construction, free-ridership occurs when participants would have still implemented energy efficiency measures in their new building in the absence of the service. The free-ridership level was assesse...

AI summary The free-ridership level for new construction was assessed at 28% using self-reported data from interviews, a decrease from 2023's 31%. The Evaluator used a 2024 questionnaire to capture influence factors, adjusting levels based on program influence. A 14% margin of error was deemed acceptable due to small sample size, with no significant difference in free-ridership between energy and peak demand savings.

Gross Savings NTGR Net Savings Realization Rate Value Unit Value Value Unit Value Energy Savings Tracked Savings by E1 20.900 GWh 0.72 14.421 GWh Evaluation Results 21.495 GWh 0.72 15.476 GWh 107% Peak Demand Savings Tracked Savings by E1...

AI summary Evaluated energy and peak demand savings were higher than tracked savings due to adjustments in energy models and slightly lower free-ridership in 2024, which increased the realization rate.

Table 26: Evaluated 2024 Building Optimization NTGR Value for Savings Claimed in 2024 Free-Ridership Level 9% (±7.2%) Participant Spillover Level 0% NTGR 0.91 Free-ridership and spillover levels were not measured in 2024, results from the...

AI summary Table 26 presents the evaluated 2024 Building Optimization NTGR, showing a free-ridership level of 9% (±7.2%) and an NTGR of 0.91. The free-ridership and spillover levels from 2021 were applied since they were not measured in 2024.

General Custom Key Findings and Recommendations 2024 Custom - Finding: Custom net electrical energy and peak demand savings surpassed targets in 2024. Custom achieved 39.951 GWh in net electrical energy savings and 5.225 MW in net peak dem...

AI summary Custom program exceeded 2024 energy and peak demand savings targets by 48% and 28%, respectively. Participation decreased in Retrofit and Building Optimization but increased in New Construction. Evaluator adjustments led to varying ARs across services. Free-ridership dropped for Retrofit and New Construction. Evaluated savings were 8-5% higher than E1's tracking.

Custom Appendix I: Retrofit Participant Interview Guide Appendix II: Retrofit Algorithm for Free-Ridership Calculation Appendix III: Retrofit Algorithm for Participant Spillover Calculation Appendix IV: Retrofit and Building Optimization T...

AI summary The document outlines appendices related to a 'Custom' regulatory process, including tools for evaluating energy efficiency programs. Key components include free-ridership and spillover calculation algorithms, participant interview guides, project review protocols, and adjustment ratio examples, emphasizing program evaluation and data collection methods.

C. Free-ridership - C1. Had your organization finalized the design and planning of the energy efficiency project BEFORE knowing that you would receive an incentive from Efficiency Nova Scotia? - 1. Yes - 2. No - 98. Don't know - 99. Refuse...

AI summary The document includes survey questions assessing free-ridership in energy efficiency projects, focusing on whether organizations finalized projects before knowing about incentives, confidence in receiving rebates, and the financial impact of incentives on project viability. It seeks to quantify how incentives influence payback periods and project implementation likelihood.

APPENDIX II Retrofit Algorithm for Free-Ridership Calculation Question Answer Score Identifying Key Decision Makers We would like to speak with someone that 1) Yes A1a played a key role in the financial decision to 2) No EMPTY implement th...

AI summary This appendix outlines a retrofit algorithm for calculating free-ridership in energy efficiency programs. It includes a series of questions aimed at identifying key decision-makers and understanding the planning process related to energy efficiency projects, including confidence levels in receiving incentives from Efficiency Nova Scotia.

Cross-Influence D6 from your organization (or a supplier/contractor) to examine the energy efficiency options for your project 98/99) Don't know/Refused EMPTY [IF D4 = 1] The Efficiency Nova Scotia 1) Agree 1 energy efficiency promotional...

AI summary The text outlines a cross-influence calculation related to energy efficiency measures, referencing Efficiency Nova Scotia and evaluating the effectiveness of promotional materials in prompting consideration of energy efficiency options. It includes conditional logic for revising free-ridership rates based on responses to various prompts.

C. Free-ridership C1. [CAPTURE VERBATIM; KEEP CODE 98 & 96 EXCLUSIVE] Why did your organization decide to build a better-than-code building? CAPTURE VERBATIM - 98. I am unsure - 99. I prefer not to say - C2. [SINGLE RESPONSE] Was the desig...

AI summary The text presents survey questions assessing free-ridership in energy efficiency programs, inquiring whether organizations constructed buildings exceeding code standards and if they knew about Efficiency Nova Scotia incentives prior to design finalization.

Table 3: Overview of Data Collection Activity Descriptor This Instrument Instrument Type Semi-directed in-depth interview Estimated Time to Complete 15-20 Minutes Target Audience › Participant Builders Expected Number of Completions › Part...

AI summary This document outlines data collection activities involving semi-directed in-depth interviews with 12 participant builders to assess free-ridership levels, program satisfaction, and barriers to participation in New Construction projects. Econoler is responsible for fielding the interviews, with an estimated timeline of November 2024.

B. Free-ridership - B1. Why did your organization decide to build a better-than-code building? [DO NOT READ. MULTIPLE RESPONSE] - 1. Lower operation costs - 2. Energy policy in my organization - 3. Environmental reasons/energy efficiency -...

AI summary The section includes survey questions about motivations for building better-than-code structures, pre-incentive design decisions, and confidence in receiving Efficiency Nova Scotia incentives, focusing on free-ridership implications in energy efficiency programs.

Table 1: Participant Interview Questionnaire and Free-ridership Algorithm Question Answer Score A4 [IF A1=NO, DK or REFUSED] Who were the key decision-makers that played a key role in the decision to build a better-than-code building? Can...

AI summary The table outlines a questionnaire for participants in a building efficiency initiative, focusing on key decision-makers and motivations for constructing better-than-code buildings. It includes questions about incentives, decision-making processes, and free-ridership algorithm context.

Note on Margins of Error For evaluation activities that yield quantitative results based on a sample, the Evaluator aimed to achieve a maximum margin of error of 10% at a confidence level of 90%. This means that, if measurements were condu...

AI summary The Evaluator aimed for a 10% margin of error at 90% confidence in quantitative evaluations. For the 2024 SBES evaluation, no margins were calculated, though 2023 free-ridership results include margins. Confidence levels exclude non-sampling errors like data entry or response biases.

4.3.1 Free-Ridership For SBES, free-ridership occurs when participants would have implemented energy efficiency upgrades in the absence of the program component. The free-ridership levels for DIY and Audit projects were assessed during the...

AI summary The document discusses free-ridership in the context of the SBES and CDI pilot programs. Free-ridership refers to participants who would have implemented energy efficiency upgrades without the program. For SBES, free-ridership levels from 2023 were used in 2024 as no new data was collected. For the CDI pilot, no free-ridership activity was conducted in 2023, leading to an assumption of no free-ridership.

Table 13: 2024 SBES Free-ridership Levels Project Path Average Free-ridership Level Margin of Error DIY (based on 2023 results) 20% 5.2% Audit (based on 2019 results) 12% 6.5% CDI Pilot - N/A 4.3.2 Participant Spillover

AI summary Table 13 presents the 2024 free-ridership levels for the Small Business Energy Solutions (SBES) program, showing a 20% average for DIY projects and 12% for audit-based projects, with respective margins of error. The CDI Pilot has no data available. The section '4.3.2 Participant Spillover' likely discusses the impact of program participation on non-participants.

4.3.4 Net-to-gross Ratio Calculation The NTGR results from the comprehensive impact evaluation performed in 2023 were used for the 2024 evaluation. The NTGR for SBES was calculated using the following equation. NTGR = (1 – % Free-ridership...

AI summary The Net-to-gross Ratio (NTGR) for the Small Business Energy Solutions (SBES) program was calculated using the equation NTGR = (1 – % Free-ridership + % Participant Spillover), based on the 2023 comprehensive impact evaluation results used for the 2024 evaluation.

Table 14: 2024 SBES NTGRs Project Path Free-ridership Participant Spillover NTGR DIY 20% 0% 0.80 Audit 12% 0% 0.88 CDI Pilot - - 1.00 4.3.5 Evaluated Net Savings

AI summary Table 14 presents 2024 SBES NTGRs for different project paths, including DIY, Audit, and CDI Pilot. The table shows free-ridership, participant spillover, and NTGR values. Section 4.3.5 discusses evaluated net savings.

5 SBES Key Findings and Recommendations The main objectives of the 2024 SBES evaluation were as follows: - › Calculate gross and net SBES results, namely electrical first-year and lifetime energy savings, peak demand savings, as well as av...

AI summary The 2024 SBES evaluation found that the program met its net electrical energy savings target (10.854 GWh, 2% over target) but missed its peak demand savings target (2.155 MW, 16% short). Evaluated savings exceeded E1's tracked results by 4% and 6% respectively. Participation rose 53% year-over-year, with DIY Path dominating. Spillover surveys found no non-participant impact.

EXECUTIVE SUMMARY EfficiencyOne (E1) commissioned Econoler to evaluate E1's 2023-2025 DSM program portfolio. As part of the evaluation scope, E1 asked Econoler to present a review of net-to-gross ratio (NTGR) best practices and update the...

AI summary EfficiencyOne commissioned Econoler to evaluate its 2023-2025 DSM program portfolio, focusing on updating net-to-gross ratio (NTGR) best practices and free-ridership methodologies. The review relied on Uniform Methods Project (UMP) guidelines and regional evaluation protocols, emphasizing consistency across programs using self-report data. Econoler identified areas for maintenance and updates to align with current best practices.

Key Changes - › Averaging the intention and influence scores to calculate free-ridership. This updated approach adopted by several jurisdictions ensures that both variables carry an equal weighting as decision-making factors. Econoler reco...

AI summary Key changes include averaging intention and influence scores for free-ridership calculation, adopting a four-point unbalanced scale for survey clarity, simplifying question wording, and ensuring NTGR consideration using deemed or program-level ratios. Econoler recommends these updates to improve accuracy and participant comprehension in E1 programs.

INTRODUCTION EfficiencyOne (E1), an independent, non-profit organization, is responsible for helping Nova Scotians improve the energy efficiency of their homes and workplaces by designing, marketing, and delivering demand-side management (...

AI summary EfficiencyOne (E1) manages Nova Scotia's demand-side management (DSM) programs, evaluated by Econoler to align net-to-gross ratio (NTGR) algorithms and free-ridership questionnaires with best practices. The review focuses on self-report methods for DSM programs, excluding non-participant spillover effects, and updates algorithms for EPI and BER-Instant Rebates.

7 Net-to-gross Ratio (NTGR) Methodology This section documents best practices and recommendations with respect to NTGR calculations, NTGR methodology, algorithm design, free-ridership (FR) and participant spillover (SO) scoring, consistenc...

AI summary This section outlines best practices for calculating the Net-to-Gross Ratio (NTGR), including algorithm design, free-ridership (FR) and participant spillover (SO) scoring, consistency checks, and study design methodologies for regulatory evaluations.

7.1.1 The Net-to-gross Ratio Calculation The NTGR is applied to a program to ensure the savings being attributed to the given program would not have happened naturally without the intervention of the program. It is best practice across all...

AI summary The Net-to-Gross Ratio (NTGR) ensures program savings are not naturally occurring. Best practices include evaluating free-ridership and spillover effects through self-reporting tools. The calculation formula is referenced but not detailed in this section.

Free-ridership Score Components In calculating free-ridership, the self-report method is aimed at establishing how the participant decided to implement the energy efficient or demand response measure(s). Three factors need to be considered...

AI summary The free-ridership score calculation uses a self-report method evaluating three factors: Intention (likelihood of implementing measures without the program), Influence (program's impact on decisions), and Cross-influence (past program efforts' effect on current decisions). These factors assess participant behavior and program effectiveness.

Partial Free-riders Free-ridership scores can vary between 0% and 100%. It is best practice to ensure free-ridership questions are nuanced to enable accounting for partial free-ridership. Partial free-riders are those who fall somewhere in...

AI summary Free-ridership scores range from 0% to 100%, with partial free-riders occupying the middle ground. Best practices involve nuanced questions to account for partial free-ridership, adjusting scores by evaluating factors like planning, cost, effort, and cross-influence.

100% score (free-rider): - › Would have installed in the same timeframe and context - › Would have installed the same quantity of measures and efficiency level - › Installed measures prior to learning about program

AI summary The text outlines criteria for identifying free-riders in programs, where participants would have installed measures regardless of the program's existence, indicating the program did not influence their actions. This relates to free-ridership measurement.

Partial free-rider: - › Program had impact on quantity/timing/efficiency of upgrades, but not full impact - › Aspects of the program influenced their decision to participate

AI summary The program had a partial impact on the quantity, timing, and efficiency of upgrades, though not a full impact. Certain program aspects influenced participants' decisions to engage, indicating partial free-rider effects rather than complete program influence.

0% score: › Would not have taken any action without the program 394 Massachusetts in 2022, New Jersey (new to DSM) in 2023, and Efficiency Maine's most recent C&I evaluation in 2023. 2024-2025 DSM Measure Assessment Final Report 327 393 Ma...

AI summary The text references DSM evaluations in Massachusetts, New Jersey, and Efficiency Maine, noting methods like cross-influence scores and free-ridership measurements. It highlights regional approaches to assessing program effectiveness and potential biases in evaluation metrics.

Free-ridership Scoring In the self-report method, an important consideration is scoring responses that are used to establish the freeridership result. The scoring of individual scale points may vary based on the algorithm, but it is best p...

AI summary The document discusses free-ridership scoring methods, emphasizing consistency in scale points and labels. E1 currently uses a 10-point scale, while Econoler recommends a four-point labelled scale for uniformity and accuracy in evaluations. The four-point scale allows incremental scoring and is adaptable to algorithms, though it may not apply to all question types.

Checking for Consistency Ensuring consistency in responses is considered best practice according to the UMP and all jurisdictions studied; this is accomplished by including a series of questions to capture any responses that contradict oth...

AI summary The text emphasizes the importance of consistency checks in free-ridership algorithms, citing the UMP as a best practice. E1's current approach is deemed sufficient, but the Evaluator recommends three enhancements: follow-up questions, response comparison, and open-ended inquiries for BNI customers with large projects.

Spillover Score Components Spillover is a component of the NTGR aimed at capturing further positive effects and is generally added to a questionnaire after free-ridership questions. The spillover series of questions is aimed at capturing o...

AI summary Spillover Score Components under NTGR aim to capture energy efficiency actions beyond free-ridership, improving NTGR if attributable to programs. Table 367 provides UMP definitions for spillover types.

7.2 NTGR Study Design and Sampling The following considerations are important when designing a study to capture the NTGR for any program. › Free-ridership should be explicitly considered for all measures in a program. Where possible, evalu...

AI summary The text outlines key considerations for designing studies to capture the Net-to-Gross Ratio (NTGR) in programs, emphasizing the need to account for free-ridership, ensure robust sampling, and apply weighted NTGR values when multiple measures are involved. Alternative NTGR approaches must be used for excluded measures.

- › The sample size should be designed to aim for a 90% confidence level and a 10% margin of error when based on the available population of projects in a program. Project-level sampling is best practice as survey questions are posed on a...

AI summary The text discusses sampling methodologies for program evaluation, emphasizing project-level sampling for 90% confidence and 10% margin of error. It addresses NTGR scoring adjustments for midstream programs, recommending inclusion of distributor influence on free-ridership, with E1's BER-IR program as a case study. Econoler advises incorporating stakeholder perspectives in NTGR scoring for accurate program evaluation.

7.3 Best Practices in Survey Design and Data Collection Although various methods[399](#page-13-0) can be used to calculate NTGRs,[400](#page-13-1) the self-report survey approach is standard practice due to its overall cost-effectiveness,...

AI summary The text outlines the use of self-report surveys for calculating NTGRs, emphasizing their cost-effectiveness and flexibility. It highlights surveying program participants and market actors to estimate free-ridership and spillover effects, with data collection methods including telephone and web surveys.

Question Design - › Use simple words and avoid technical terms and slang - › Use specific, clear wording rather than general, abstract terms - › Use words that can be interpreted in only one way to avoid ambiguity - › Use question wording...

AI summary The document outlines principles for designing effective survey questions in regulatory proceedings, emphasizing clarity, neutrality, and methodological rigor. It references academic studies and frameworks like the Uniform Methods Project for evaluating energy efficiency programs, billing analysis, and free ridership estimation.

Table 369: Summary of E1's Current NTGR Approach Program Component Measures for Which FR Is Established Year of Last FR Update Current Spillover Methodology Year of Last Spillover Update Proposed Changes GH FR determined for all measures....

AI summary Table 369 outlines E1's current approach to Net-to-Gross Ratio (NTGR) for free-ridership (FR) and spillover. FR is established for all measures, with NTGR categorized by type. Spillover methodology is not currently applied due to limited opportunities. A proposed change involves collecting additional information on spillover for completeness.

Aspects Maintained - › The overall adjustment approach to cross-influence currently used in E1 algorithms. This approach captures influences outside the program (e.g. past participation, educational initiatives, etc.), the results of which...

AI summary The text outlines maintained aspects of E1 algorithms' cross-influence adjustments, spillover capture methods, consistency checks for NTGR calculations, and questionnaire design for program evaluations. These elements aim to ensure accurate free-ridership measurement and program effectiveness.

Key Changes - › Averaging the intention and influence scores to calculate free-ridership. This updated approach adopted by several jurisdictions ensures that both variables carry an equal weighting as decision-making factors. Econoler reco...

AI summary The text outlines key changes to free-ridership measurement methods, including averaging intention and influence scores, adopting a four-point unbalanced scale, simplifying survey questions, and ensuring NTGR consideration. Econoler recommends these updates to improve accuracy and participant comprehension in E1 programs like EPI.

9.1.1 Efficient Product Install Algorithm Below is an example of the free-ridership algorithm for the measure that generates the highest savings in EPI: LED lamps. Please see Section 9.1.3 for an example calculation. Econoler will adapt th...

AI summary The text outlines the approach for calculating free-ridership in the Efficient Product Installation (EPI) program, specifically for LED lamps. Econoler will adapt the algorithm for other high-saving measures like smart thermostats, while using a deemed NTGR for low-saving measures based on historical data or research from other jurisdictions.

Table 370: Free-Ridership - LEDs Current Al gorithm Adjuste d Algorithm Question Answer Score Question Answer Score INTE NTION Planr ning Had you already decided to purchase 1) Yes Use E2 Before learning about the Efficient 1) Yes Use E2 E...

AI summary This table discusses a free-ridership analysis related to LED bulbs, focusing on whether participants had already decided to purchase and install LED bulbs before learning about the Efficient Product Installation Service. It includes questions, answers, and scores for both the current and adjusted algorithms.

Final Report 341 Current A lgorithm Adjuste d Algorithm b) Information or advice provided by the service staff 97/98/99) Not applicable/ Don't know/Refused b) The information or advice provided by the service staff 98/99) Don't know/Refuse...

AI summary The text presents a table comparing a current algorithm and an adjusted algorithm, focusing on scoring methods related to free-ridership and influence scores. The table includes questions and answers, with scoring criteria such as 'INFLUENCE Score (PA5)' and 'Free-Ridership (FR)' being calculated using different formulas under the two algorithms.

Table 373: Instant Rebates Distributor Influence Level Algorithm Current Algorithm Adjusted Algorithm ised Participant Ridership: CI = 0 OR 1: Revised FR = Part FR CI = 2: Revised FR = Part FR 0.75 CI = 3: Revised FR = Part FR 0.50 Revised...

AI summary Table 373 outlines the Instant Rebates Distributor Influence Level Algorithm, which adjusts free-ridership (FR) calculations based on the Cross-Influence (CI) level. The algorithm modifies the Revised FR for both participants and end users depending on whether CI is 0, 1, 2, or 3.

Below is a calculation example of the adjusted free-ridership algorithm for LED lamps in EPI for one participant's responses (responses indicated by green highlight).

AI summary This text provides an example of calculating the adjusted free-ridership algorithm for LED lamps in the Efficient Product Installation (EPI) program based on a participant's responses highlighted in green.

Table 375: Free-Ridership – LEDs Adjusted Algorithm Question Answer Score 1) Yes Use E2 E1 Before learning about the Efficient Product Installation Service, had you already decided to 2) No purchase and install LED bulbs in your home? 98/9...

AI summary This table examines free-ridership related to LED bulb installations through the Efficient Product Installation Service. It asks respondents whether they would have purchased and installed LED bulbs without the service, with 25% indicating they probably would not have.

April 2016). Gross Savings and Net Savings: Principles and Guidance. [https://neep.org/gross-and-net-savings-principles-and-guidance.](https://neep.org/gross-and-net-savings-principles-and-guidance) PWP, Inc Evergreen Economics for Energy...

AI summary The text lists references to energy efficiency studies and guidelines, including methodologies for evaluating free ridership, spillover effects, and net-to-gross ratios in rebate programs. Sources include reports from organizations like Synapse Energy Economics and the New Jersey BPU.

ch-](https://www.synapse-energy.com/sites/default/files/NTG-Research-14-053.pdf)[14-053.pdf.](https://www.synapse-energy.com/sites/default/files/NTG-Research-14-053.pdf) Tetra Tech for National Grid, NSTAR, Western Massachusetts Electric C...

AI summary The text references multiple studies on free-ridership and spillover effects in commercial and industrial energy efficiency programs conducted by entities like Tetra Tech, National Grid, and the California Public Utilities Commission. These studies analyze program impacts and evaluation methodologies, with links to technical reports and conference papers.

Evaluator Findings. The Evaluator reported the following key Instant Savings findings: - Instant Savings exceeded both 2024 planned net electrical energy and peak demand savings of 12.544 GWh and 1.680 MW, respectively. - 2024 net electric...

AI summary Instant Savings exceeded 2024 energy and peak demand savings targets by 77% and 45%, driven largely by ENERGY STAR LED products. Non-lighting savings increased 17% YoY, and free ridership for LEDs dropped from 59% to 39%. Evaluated savings were 16-10% higher than Efficiency Nova Scotia's tracked values. The evaluation methodology was deemed appropriate and excellent.

This program serves the business, non-profit and institutional sector (BNI sector). The Efficient Product Rebates program operates the Business Energy Rebates (BER) program. This program has two components, Instant Rebates at point-of-sale...

AI summary The Business Energy Rebates (BER) program, part of the BNI sector, provides rebates for energy-efficient products. In 2024, point-of-sale rebate evaluations showed a 7.7% decline in savings compared to 2023, with LED Linear Lamps experiencing a 41% decline. The program also conducted evaluations and provided nine recommendations to improve delivery, tracking, and marketing, including determining a new baseline for LED products.

ova Scotia reports 22 new participants in 2024 for the Pay-for-Performance program but, due to multi-year savings progression, no savings for these participants are expected to be reported until 2025. Custom/New Construction . The New Cons...

AI summary The BNI Custom Incentives Program saw 27 completed New Construction projects in 2024, with a decline in free-ridership from 39% in 2022 to 28% in 2024. The program's evaluation highlights improved effectiveness, including higher participant satisfaction and better savings tracking. However, participation rates were affected by increased finance costs and labor shortages.

Table 10: Evaluation Questions - Summary Table. Asked and Answered for Program Year 2024 General Questions to Ask of Energy Efficiency Program Evaluations 17 For programs that require on-site visits, are there enough on-site visits? Adequa...

AI summary The summary table evaluates energy efficiency programs for the year 2024, addressing questions related to program implementation and evaluation. It highlights that most programs meet evaluation standards, though some areas like metering accuracy and free-ridership calculations could be improved.