Topic:"Program Evaluation" in M08604

Page i Efficiency Nova Scotia 2017 DSM ANNUAL PROGRESS REPORT 1 5.1 Locational DSM Efforts ..................................................................................................................... 63 2 5.2 Historical Rate and B...

AI summary Efficiency Nova Scotia's 2017 DSM Annual Progress Report outlines locational DSM efforts, historical rate and bill impact analysis, cost-effectiveness testing, incentive methodology implementation, and updates on evaluation recommendations. The report covers key aspects of demand-side management programs and their financial implications.

% 405.9 GWh 62.4 MW 90% 80% 80% 23.7 MW 70% 66% Percent of 2016-2018 Target 130.7 GWh 60% 50% 40% 26.0 MW 30% 136.9 GWh 20% 10% 0% Energy savings (2016-2017 Energy savings (2016-2018 Peak demand savings (2016- Peak demand savings (2016- ac...

AI summary The text presents energy and peak demand savings data for 2016-2018, noting that 2016 savings are verified while 2017 are pending. It mentions free ridership and spillover effects in the evaluation of energy savings, with a focus on measurement and evaluation processes.

ator and are net of free ridership and spillover. 2016 3 savings are evaluated and verified; 2017 savings are evaluated and pending verification. 4 5 Measurement and Evaluation 6 7 In 2017, ENS contracted two consultants to perform evaluat...

AI summary In 2017, Efficiency Nova Scotia (ENS) evaluated DSM program savings, including 21.9 GWh energy and 4.0 MW peak demand savings from Codes and Standards. Evaluations excluded Appliance Retirement (no market evaluation) and Strategic Energy Management (no savings claimed). Two consultants assessed programs, with one focusing on DSM and the other on process evaluations.

ement, which did not receive an impact or market 6 evaluation since it did not claim any savings in 2017 (please refer to section 1.3.2 7 for further discussion). 8 9 Impact evaluations assess net incremental energy and net peak demand sav...

AI summary The text discusses program evaluations, noting reduced impact and market evaluations due to stable parameters. Process evaluations were conducted for Green Heat and Custom Retrofit programs, along with support processes for marketing and incentive-setting practices under ENS's guidelines.

o Business development; and 26 o Incentive setting (in accordance with ENS’s Incentive Setting 27 Implementation Plan). 5 This is as a result of consistent participant survey results in previous years due to a stable market and the maturit...

AI summary Efficiency Nova Scotia (ENS) reports on 2017 DSM program evaluations, noting reduced impact evaluations for Appliance Retirement, Instant Savings, Business Energy Rebates, and Custom programs due to market stability and program maturity. Expenditures for 2017 are unaudited, with figures presented in the report.

usinesses. 7 Awareness of, and participation in, programs and services was facilitated through 8 advertising, media engagements, social media, and community outreach initiatives. 9 10 ENS also continued working on a number of pilot project...

AI summary Efficiency Nova Scotia (ENS) promoted energy programs via media and outreach, maintained a 31% unaided awareness rate, achieved 89.7 Customer Satisfaction Index, and implemented Dynamic DSM (DDSM) for improved data tracking. Pilot projects in residential and BNI sectors aimed to diversify program offerings and measure market needs.

1 result, dehumidifiers and air purifiers will be added to the list of energy-efficient 2 products available in Instant Savings for 2018. 3 4 Appliance Retirement 5 6 In 2017, the Appliance Retirement component of the EPR program achieved...

AI summary The text discusses energy efficiency achievements in 2017, including 3.1 GWh energy savings from Appliance Retirement and 27.9 GWh from Existing Residential programs. It highlights a pilot project with the Halifax Regional School Board (HRSB) retiring inefficient appliances and replacing them with efficient models via HomeWarming. ENS collaborated with HRSB on energy savings communication and coordination.

enditures reported in previous 4 quarters may have been updated and are reflected in the year-end totals. 5 New Residential includes the New Home Construction program component. 6 7 NHC saw an increase in registrations in 2017 compared to...

AI summary The New Home Construction (NHC) program saw increased registrations in 2017 but fell short of planned energy savings due to a large Service Provider exiting the program. Efficiency Nova Scotia (ENS) assisted in transitioning participants to other providers and updated incentives in Q2 2017 to address feedback on the need for higher incentives.

6 initiative in 2017. Under this initiative, ENS places an energy manager within a large 27 organization to identify new energy-efficiency projects and to support the organization 19 New Home Construction Program, 2015 DSM Evaluation: Effi...

AI summary Efficiency Nova Scotia (ENS) implemented an initiative in 2017 placing energy managers in large organizations to identify energy-efficiency projects. The BNI sector achieved higher net peak demand savings than energy savings in 2017, partly due to mid-course adjustments. Efficient Product Rebates (BER) achieved significant energy and peak demand savings in 2017.

7 In 2017, Regulatory Affairs activities included: 8 • engaging with regulatory stakeholders, the DSM Advisory Group and consultants, 9 as appropriate, on topics including: 10 o the development approach for the 2019 DSM Resource Plan; 11 o...

AI summary In 2017, Regulatory Affairs engaged with stakeholders on topics including the 2019 DSM Resource Plan, non-energy benefits quantification, and the Incentive Setting Methodology Study. Activities included UARB reporting, stakeholder meetings, responding to information requests, and conducting audits related to EfficiencyOne and program controls.

. 30 ENS will provide its mid-course adjustments for 8 2018 by program and their customer rate class impacts within its 2018 Q1 DSM Report, 9 due to be filed in May, 2018. 10 11 In 2018, ENS will work with its Evaluation Consultant to desi...

AI summary ENS plans to provide mid-course adjustments for 2018 by program and customer rate class impacts in its Q1 DSM Report, due in May 2018. It will continue conducting evaluations and process improvements to enhance program effectiveness and participant experience.

CLEAResult’s 17 recommendations. The incentive setting and review process uses a Consolidated 18 Calculator, which was developed by ENS further to recommendations from CLEAResult. 19 20 ENS also performed a financial simulation for all mea...

AI summary Efficiency Nova Scotia (ENS) implemented incentive setting and review processes based on CLEAResult’s recommendations, including a Consolidated Calculator and financial simulations. An evaluation of ENS’s progress was conducted and filed as part of the 2017 Evaluation Report, with plans to refine processes further. ENS anticipates meeting 2018 savings targets and continuing DSM programs.

ooks forward to the opportunity to respond to questions on this report and to continue 7 updating the UARB and stakeholders on its progress throughout the remainder of the 8 year. DATE FILED: March 29, 2018 Page 68 of 68 ATTACHMENT 1 EVALU...

AI summary The document outlines updates on the implementation of evaluation and verification recommendations from 2013 to 2015, with tables providing detailed progress on each recommendation.

Attachment 1, Page 1 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation. It outlines progress made and any remaining actions required to address the findings of the evaluation.

Expected Evaluation/ Year Recommendation Text Source Status Comments Period of Verification Completion 2013 Evaluation Continue to work on including as much information as possible in OV-R6 In Progress ENS agrees with this recommendation a...

AI summary In 2013, a recommendation was made to improve the Dynamic DSM data tracking system by including more standardized information in the DSMDS. ENS agreed and continued implementing the system, which improved customer experience and program delivery efficiency by 2017.

Attachment 1, Page 2 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, likely related to energy efficiency or regulatory processes in Nova Scotia.

Attachment 1, Page 3 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, indicating progress and ongoing efforts to address identified issues.

ional quality control steps be defined and implemented by the program manager to ensure that savings are calculated accurately upon project approval. Attachment 1, Table 1, 2013-2015 Evaluation Recommendations Efficiency Nova Scotia - 2017...

AI summary The text discusses the need for quality control steps to be defined and implemented by the program manager to ensure accurate calculation of savings upon project approval. It references an evaluation report and a table from an attachment.

Attachment 1, Page 4 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, focusing on progress made and any outstanding issues that require further action.

2013 Verification Research Questions. The next process evaluation should begin not at the level of official Process Evaluation Complete ENS agrees with this recommendation. Its Evaluator Complete documents, the expectations of executive ma...

AI summary The text discusses the process evaluation of Efficiency Nova Scotia (ENS), emphasizing the need for research questions that focus on practical and empirical perspectives rather than just official documents. ENS agrees with the recommendation and has developed research questions for its process evaluations to assess how well its processes function in practice.

s responsibility for service relationships with the people and institutions that ENSC serves? What are the restrictions on scope and to what extent are they needed or unnecessary? How and how well does ENSC maintain good faith and quality...

AI summary The text explores the service quality and organizational culture of ENSC, focusing on how well it maintains good faith and quality of service in customer interactions, potential unintended consequences of its work, and the internal and external experiences of employees and former employees. It also raises questions about the causes of underservice and how programs have adapted to unforeseen side effects.

2013 Verification Boundary for Analysis. The next process evaluation should draw the boundary for the study Process Evaluation Complete ENS agrees with this recommendation and has Complete around ENSC plus its service delivery agencies and...

AI summary The document discusses a process evaluation conducted by ENS, which agrees with the recommendation to expand the boundary of analysis to include ENSC and its service delivery agencies. The evaluation involved interviews and literature review, and ENS plans to include external partners in future evaluations.

uld be inclusive of all staff levels, delivery agent staff and Process Evaluation Complete ENS agrees with this recommendation and has Complete others who do the work. SV-4 implemented it in the Marketing evaluation and Practices & Procedu...

AI summary The document discusses the implementation of process evaluations by ENS, emphasizing the inclusion of all staff levels and partners. ENS agrees with the recommendation and has already implemented it in previous evaluations, ensuring comprehensive feedback through interviews and surveys.

interviews/surveys with all relevant groups involved. 2013 Verification Confidentiality. Rather than focus groups, group settings or surveys which require personal Process Evaluation Complete ENS agrees with this recommendation. The Evalua...

AI summary The text discusses the approach taken by ENS in conducting evaluations, emphasizing the use of individual interviews and maintaining confidentiality. The evaluator engaged in dialog with individuals and ensured that respondent identities were not disclosed unless required by law.

. A separate document from a TRM is a set of evaluation Savings In Progress ENS agrees with this recommendation and is 2018 guidelines covering methods, practice and reporting. The New York evaluation guidelines and Verification developing...

AI summary The text discusses the development of evaluation guidelines, referencing the New York evaluation guidelines and appendices. It mentions that ENS agrees with a recommendation and is developing its own guidelines to set standards for future evaluations.

ndation approach and expectations for future evaluations. certain evaluation standards for future evaluations. No. 3 2014 Verification Savings Weighted Measure Lives. The time dimension should be operationalized in the Savings Complete ENS...

AI summary The document discusses the need for a more comprehensive evaluation system for energy efficiency programs, emphasizing the importance of using lifetime energy savings rather than just first-year savings. ENS agrees with the recommendation and began reporting lifetime energy savings in 2017. The Evaluator reviewed ENS's methodology and used a multiple baseline approach to determine lifetime energy savings.

conduct a future calibration billing analysis. 2015 Verification Savings Weighted Measure Lives. The time and savings dimension should be operationalized SV-2 Complete ENS agrees with this recommendation and began Complete in the indicator...

AI summary The text discusses the need for a more comprehensive approach to evaluating energy savings by incorporating a weighted measure of lives, rather than relying solely on first-year energy savings. ENS has started reporting lifetime energy savings for its programs, and the Evaluator has reviewed this methodology using a multiple baseline approach to determine lifetime energy savings.

still likely to start installing some of these measures without receiving any incentive encouraging them to do so. Therefore, the Evaluator recommends starting to measure spillover for RP&C in 2017. Continue conducting project reviews to a...

AI summary The Evaluator recommends measuring spillover for RP&C starting in 2017 and continuing project reviews to assess simulation file quality. ENS agrees and conducts on-site visits and HOT200 simulation reviews, noting that EAs generally perform well but recommends regular reviews to maintain simulation quality. Recurrent mistakes in HOT2000 simulations are noted and will be addressed in the 2017 evaluation.

nd will follow up on these aspects during the 2017 evaluation. In the meantime, the Evaluator recommends sharing with EAs those aspects of the project review process that ENS would like to improve.

AI summary The Evaluator suggests that Energy Advisors (EAs) be informed about aspects of the project review process that ENS would like to improve, with follow-up planned for the 2017 evaluation.

oping such M&V procedures for Retrofit projects. The same procedures should be applied to BER projects generating 300,000 kWh and more to limit the corrections needed to the tracked energy savings. Conduct deeper market analysis to adapt t...

AI summary The document discusses the need for updated M&V procedures for Retrofit and BER projects and the importance of deeper market analysis to adapt BER Instant Rebates. It highlights market evolution, reduced barriers, and the need to assess when certain products should be removed from the Instant Rebates service due to market transformation.

Completion Track incremental savings rather than cumulative savings. For continuing participants in 2016, the Evaluator SEM-R4 In Progress ENS agrees with this recommendation. There were no 2017 SEM savings but 2018 modified the methodolog...

AI summary The document discusses the methodology for tracking energy savings under the SEM program, emphasizing the use of incremental rather than cumulative savings. ENS agrees with modifying the approach to align with incremental first-year savings and acknowledges the need for ongoing support to maintain savings. The methodology may be refined as more research becomes available.

pport is available in the technical literature. As new research is conducted on the impact evaluation of programs similar to SEM, this methodology might be refined to capture all incremental savings. Improve the identification of event typ...

AI summary The text discusses improvements to the Event Log for better impact evaluation of the SEM program, suggesting more specific event categories. It also highlights the need to identify energy efficiency measures installed under ENS instant rebate programs to avoid double-counting savings.

ny associated savings provide invoices of energy efficient products bought under ENS instant rebate programs. are deducted from SEM savings. Apply the adjustments made to the baseline energy regression in the 2016 evaluation to continuing...

AI summary The document discusses adjustments to baseline energy regression for continuing projects in the SEM program. It notes that adjustments made in the 2015 evaluation were valid for 2016 but not implemented by the service provider. ENS agrees with applying these adjustments to continuing participants in SEM.

ants that take part in SEM for more than one year. Therefore, the Evaluator recommends that the project-specific adjustments made in this evaluation be applied to continuing projects going forward.

AI summary The Evaluator recommends applying project-specific adjustments from this evaluation to continuing projects, particularly those involving participants in SEM for more than one year.

Attachment 1, Table 3, 2016 Evaluation Recommendations Efficiency Nova Scotia - 2017 DSM Annual Progress Report Date Filed: March 29, 2018 Table 4 Attachment 1, Page 17 of 20 Update on Implementation of 2016 Verification Recommendations Ex...

AI summary The document references the 2016 Savings Verification study, which recommends accepting the 2016 evaluation results for energy efficiency programs. No action is required for this recommendation.

Completion The Savings Verification study recommends acceptance of the 2016 evaluation results for energy Recommendation N/A No action required. N/A savings and for demand-reduction for all programs. General-SV-1

AI summary The Savings Verification study recommends accepting the 2016 evaluation results for energy savings and demand reduction across all programs, with no further action required.

In future years, ENS should require the Evaluator to include a copy of the operative evaluation plan Recommendation Complete ENS agrees with this recommendation. In the 2017 DSM Programs Complete within the overall Executive Summary, suppl...

AI summary ENS agrees with a recommendation to improve the evaluation process by including a copy of the operative evaluation plan in the Executive Summary and explaining the rationale for changes in evaluation components. The 2017 DSM Programs evaluation included a discussion of evaluation methods and changes from 2016.

logy section of the corresponding program component report. In future years, ENS should require the Evaluator to provide a summary table of obtained confidence Recommendation Complete In each evaluation report's Methodology section, the Ev...

AI summary The document outlines requirements for the Evaluator to include detailed methodology and precision levels in evaluation reports, including confidence levels, sample sizes, and explanations for variations in results. It also references example calculations provided in an appendix of a previous evaluation report.

x II of the 2017 DSM Programs Evaluation Executive Summary. In future years, ENS should specify rules to bound the dropping of evaluation components and to Recommendation In Progress ENS agrees with this recommendation. ENS has now complet...

AI summary The document discusses the need for ENS to establish rules to limit the reduction of program impact evaluations and ensure consistent, transparent evaluation planning. ENS agrees with the recommendation and plans to refine its approach and finalize guidelines by 2018.

that take into account lessons learned during the 2017 evaluation process. carefully watched and rules set up to protect the continuing validity and precision of evaluation results. ENS should increase on-site inspection for quality contro...

AI summary The document discusses the need for ENS to increase on-site inspections for quality control, particularly for the Small Business Energy Solutions (SBES) program. It also highlights the importance of considering lifetime savings estimates in evaluations to improve program comparisons and foresight.

viding savings estimates based on expected lifetimes to support comparisons General-SV-1 in each of its 2017 evaluation reports. among programs and to improve foresight. For program types that permit, ENS and the Advisory Group should cons...

AI summary The text discusses the evaluation of energy efficiency programs, focusing on the use of savings estimates based on expected lifetimes to compare programs and improve foresight. It also mentions considerations regarding the potential benefits and costs of introducing persistence evaluations to document lifetimes and persistence of savings.

y Group should consider the potential benefits and costs involved to employ a Consideration Complete ENS appreciates this consideration and discussed it with the Verification Complete market transformation framework. This would apply to a...

AI summary The document discusses the potential benefits and costs of employing a market transformation framework for a limited subset of programs. ENS has engaged with various parties to evaluate program components and will continue monitoring market indicators for key product types.

For A-type LEDs, the 2017 Instant Savings Program Evaluation ENS should consider the costs and Instant-SV-R1 Complete ENS appreciates this consideration and discussed it with the Verification Complete benefits of using a market transformat...

AI summary The 2017 Instant Savings Program Evaluation by ENS should consider market transformation rather than resource acquisition, including analysis of market effects and the S-curve for A-type LEDs. Discussions with the Verification Consultant, Evaluator, and DSMAG informed the evaluation, which also included monitoring future market impacts and technology diffusion.

in the Verification Consultant's Recommendation EMIS-SV- 2. One way to more clearly communicate the approach is to show the results for each step and each EMIS-SV-3 Complete ENS agrees with this recommendation. To help communicate results...

AI summary The Verification Consultant recommends including a detailed table in the 2017 Custom Incentives (EMIS) evaluation report to clearly communicate the methodology and results of energy savings. The table should include columns such as projected baseline, adjustments, actual energy usage, and incremental savings. ENS agrees with the recommendation, and the Evaluator added summary tables to the report.

in the report and protection of participant information. Add the following SEM related terms to the Definitions (p. iii); Baseline regression, Projected baseline, SEM-SV-R1 In Progress ENS agrees with this recommendation. SEM did not achie...

AI summary The document discusses the need to update definitions related to SEM (Sustainable Energy Management) in the report, including terms like baseline regression, projected baseline, adjusted baseline, cumulative SEM/EMIS savings, and incremental SEM/EMIS savings. ENS agrees with the recommendation, but SEM did not achieve savings in 2017 and 2018, leading to the absence of a 2017 evaluation report.

Provider recommendation of a longer engagement with SEM participants. SEM-SV-R2 Deferred At this time, ENS does not have all of the information it needs to perform a To be full analysis of the cost-effectiveness of extending the SEM cycle....

AI summary The document discusses ENS's consideration of extending the SEM cycle and the need for more detailed communication of results through tables showing projected baseline, adjustments, and incremental savings. ENS agrees with the recommendation but has deferred the decision due to insufficient information.

CM 2018 SEM evaluation report. programs. Presumably such a table would also make clear how the value of 0.463 “Adjustment Ratio for Energy Savings” was calculated. Edits/Corrections: On P. 55, Figure 11, should the 2015 participant bar be...

AI summary The text discusses edits and corrections to the 2018 SEM evaluation report, including clarification on the number of participants in 2015 and a correction to a bullet point on page 64, referencing Table 25.

eenhouse gas LED Light-emitting diode MEPS Minimum energy performance standards MSHP Mini-split heat pump M&V Measurement and Verification NRCan Natural Resources Canada NTGR Net-to-gross ratio PCF Peak coincidence factor Project No. 6138...

AI summary The document is an executive summary evaluating Efficiency Nova Scotia's 2017 Demand-Side Management (DSM) programs, focusing on their effectiveness and compliance with energy efficiency goals.

savings ratio, etc.) validated or measured during the evaluation process. Unless otherwise specified, the evaluated savings are the first-year savings. Project No. 6138 iii Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive S...

AI summary The document outlines the evaluation of Efficiency Nova Scotia's 2017 demand-side management (DSM) programs, focusing on validated savings metrics such as net-to-gross ratio. The evaluation prioritizes first-year savings unless otherwise specified, emphasizing the importance of accurate measurement and verification processes.

The evaluation plan summarizes the program description, objectives, base Evaluation plan cases, documentation, evaluation methodology, budget and schedule. The first full year of energy savings generated by an energy efficiency First-year...

AI summary The text defines key metrics for evaluating energy efficiency programs, including first-year savings, free-ridership, gross energy savings, induced consumption, interactive effects, internal spillover, in-service rate, and lifetime energy savings. These terms are part of an evaluation plan to assess program effectiveness and measure energy savings.

e.g., one-ton capacity Unitary savings of an air-source heat pump) or a participant (e.g., one participant taking part in a behaviour-based program). Project No. 6138 v Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive Summa...

AI summary The text references an evaluation of Efficiency Nova Scotia's 2017 DSM programs, including examples like air-source heat pump savings and behavior-based participant engagement. Project No. 6138 is cited, focusing on program performance metrics.

TABLE OF CONTENTS INTRODUCTION ................................................................................................................................... 1 1 EVALUATION OBJECTIVES AND SCOPE ...........................................

AI summary The document outlines the structure of an evaluation report, detailing objectives, methodology (including interviews, surveys, on-site visits, and analysis techniques), and impact evaluation results. It focuses on assessing program effectiveness through data collection and analytical methods.

oided GHG Emissions Quantification............................................................................ 13 3 IMPACT EVALUATION RESULTS ............................................................................................ 14 3...

AI summary The document evaluates the impact of Demand-Side Management (DSM) programs in Nova Scotia, analyzing residential and business initiatives, codes, standards, net-to-gross ratios, lifetime energy savings, and greenhouse gas reductions. It assesses portfolio performance, market evolution, and provides recommendations for DSM effectiveness.

LIST OF TABLES Table 1: Types of Evaluations Conducted for Each Program Component ............................................. 4 Table 2: 2017 Interviews Completed .............................................................................

AI summary The document lists tables from a 2017 evaluation report covering program components, energy savings, GHG emissions, free-ridership, and recommendations. Tables track interviews, surveys, on-site visits, energy savings metrics, NTGRs, and crosscutting program recommendations, likely related to Efficiency Nova Scotia (ENS) initiatives.

........................... 37 Table 15: Recommendations for Appliance Retirement ....................................................................... 39 Table 16: Recommendations for Instant Savings .......................................

AI summary The document outlines recommendations for various energy efficiency programs evaluated by Efficiency Nova Scotia, including appliance retirement, home energy assessments, and business rebates. It references Project No. 6138 and the evaluation of 2017 DSM programs, highlighting initiatives under demand-side management and energy efficiency.

838 GWh. In addition to the savings generated by ENS DSM programs, Codes and Standards were found to have achieved energy savings of 21.914 GWh and peak demand savings of 4.028 MW at the generator2. 1 Throughout this report, energy savings...

AI summary This report evaluates ENS's 2017 DSM programs, noting 838 GWh in energy savings and 21.914 GWh from Codes & Standards (excluded as not ENS-conducted). It outlines evaluation objectives, methodology, impact results, and recommendations for program improvements.

Executive Summary 1 EVALUATION OBJECTIVES AND SCOPE The objectives of the 2017 evaluation of ENS DSM program components were to: › Assess the effectiveness of program component design and delivery › Determine the extent to which program co...

AI summary The 2017 evaluation of ENS DSM program components aimed to assess design effectiveness, implementation progress, energy savings, NTGR, and GHG emissions. The Evaluator introduced two new evaluation items: lifetime energy savings and avoided GHG emissions linked to electricity savings. Table 1 outlines programs evaluated in 2016 and 2017, with 'C' and 'R' indicating evaluation scope.

t is comprised of two services, Mail-in and Instant Rebates. In Table 1, “C” represents a complete evaluation, and “R” indicates that reduced evaluation activities were performed compared to 2016. Project No. 6138 3 Evaluation of 2017 DSM...

AI summary The text outlines the evaluation of 2017 DSM programs by Efficiency Nova Scotia, noting differences in evaluation scope between 2016 and 2017. Table 1 categorizes evaluations as complete ('C') or reduced ('R') for each program component.

ocess evaluation specifically targets the Retrofit service of the Custom program component. Due to the start date of the latest Strategic Energy Management (SEM) cohort, SEM did not have any participants who achieved savings in 2017; there...

AI summary The document discusses the 2017 evaluation of the Retrofit service in the Custom program and the Green Heat program. Process evaluations were prioritized due to changes in the Green Heat program, including the introduction of mini-split heat pumps (MSHPs) and high free-ridership. The evaluation aimed to assess the effectiveness of redesigned processes and marketing strategies.

re about the effectiveness of the program changes. Conducting an in-depth process evaluation and market evaluation helped identify ways to improve the program component design and marketing strategy. Project No. 6138 4 Evaluation of 2017 D...

AI summary The evaluation of 2017 DSM programs by Efficiency Nova Scotia (ENS) highlights the need for process and market evaluations to improve program design and marketing. Retrofit under the Custom program component contributes significantly to BNI net savings, but ENS must innovate to maintain participation as simpler projects shift to Business Energy Rebates.

t and impact evaluation activities to be conducted in 2017, the Evaluator considered the research themes that required deeper analysis and those that have been fully addressed by previous evaluations. For instance, the 2016 evaluation reve...

AI summary The 2017 evaluation focused on refining metering data for Appliance Retirement due to high error margins, while removing participant surveys to reallocate resources. For Instant Savings, LED and appliance savings were prioritized, with stable products using 2016 data. The impact evaluation scope was reduced due to unchanged NTGR metrics.

l applied to 2017. The Evaluator determined that the unitary savings values recommended in 2016 remain valid since they were based on regulations still in effect and a thorough review of hours of use. The full process evaluation conducted...

AI summary The evaluation of 2017 DSM programs found that unitary savings values from 2016 remain valid due to stable regulations and usage data. The Custom Retrofit service's impact evaluation omitted free-ridership surveys due to participant fatigue, relying instead on stable NTGR values (0.90–0.92). Other program components underwent full market and impact evaluations.

Executive Summary 2 EVALUATION METHODOLOGY This section presents the methodology used and the activities carried out to evaluate ENS DSM program components and services. The Evaluator first reviewed all relevant documentation, particularly...

AI summary The evaluation of ENS DSM programs involved reviewing documentation, conducting interviews with stakeholders, and analyzing quantitative data with a 10% margin of error. The methodology included calculating margins of error for surveys, installation rates, and energy savings, with examples provided in Appendix II.

tions. Between August 2017 and January 2018, the Evaluator conducted 112 interviews with various stakeholders. Summaries of completed interviews for each program component are presented in Table 2. Project No. 6138 7 Evaluation of 2017 DSM...

AI summary The evaluation of 2017 DSM programs by Efficiency Nova Scotia involved 112 stakeholder interviews between August 2017 and January 2018, with summaries detailed in Table 2. The process focused on assessing program components through stakeholder feedback.

- - - 28 Total 10 38 24 8 4 28 Project No. 6138 8 Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive Summary 2.2 Surveys Using contact information provided by ENS, CRA conducted phone surveys with participants and builders, a...

AI summary Efficiency Nova Scotia (ENS) evaluated its 2017 demand-side management (DSM) programs through surveys conducted by Corporate Research Associates (CRA). Surveys targeted participants, builders, and LED buyers, assessing spillover effects, free-ridership, participation barriers, and program satisfaction. An intercept survey also evaluated the Instant Savings program's effectiveness.

-store interviews were conducted at five of the participating retailers in September and October 2017. The intercept survey was meant to generate feedback on the following aspects of Instant Savings: › LED lamp purchases and expected uses...

AI summary The text describes interviews with retailers and an intercept survey evaluating 2017 DSM programs, focusing on LED lamp purchases, Instant Savings awareness, ENS, free-ridership, and demographics. Table 3 summarizes survey participation data for program components.

524 100 59 91 participants from the old path and 50 participants from the new path. 70 Mail-in participants and 60 Instant Rebates participants. 2.3 On-site Visits For the 2017 evaluation, the Evaluator visited a total of 309 participant s...

AI summary The 2017 DSM program evaluation involved 309 on-site visits to assess installation rates, technical characteristics, and savings calculations. Visits occurred across seasons, with specific focus on Efficient Product Installation, Small Business Energy Solutions, and Appliance Retirement programs, including recycling facility inspections.

34 Small Business Energy Solutions 55 Total 309 2.4 Unitary Savings Review For the evaluated program components where savings were established on a unitary basis, the Evaluator reviewed the savings presented in the tracking system. First,...

AI summary This section discusses the Unitary Savings Review process, where the Evaluator validated and updated savings data based on previous evaluations, technical references, and product specifications. It outlines methodologies used for different program components, including Efficient Product Installation and Instant Savings.

ology. For the Instant Rebates service and some measures in Instant Savings, the unitary savings from the 2016 evaluation, which had been determined using this methodology, were applied again in 2017. 2.5 Project Files Review For those pro...

AI summary The evaluation of the 2017 DSM programs involved reviewing project files and applying unitary savings from the 2016 evaluation. Adjustments were made to savings calculations and extrapolated to similar projects, particularly in Custom and Small Business Energy Solutions and Green Heat.

6.328 Due to the start date of the latest Strategic Energy Management (SEM) cohort, SEM did not have any participants who achieved savings in 2017; therefore, no evaluation was conducted for 2017. Project No. 6138 15 Evaluation of 2017 DSM...

AI summary The 2017 DSM program evaluation did not include Strategic Energy Management (SEM) due to the lack of participants achieving savings in that year. The document outlines the evaluation of 2017 DSM programs and includes a table showing peak demand savings at the generator level.

savings values are both 2 percent lower than those tracked by ENS. This difference is explained by a decrease in the NTGR value due to a reduction in 2017 spillover level compared to 2016. Green Heat The 2017 participation level for Green...

AI summary The Green Heat program saw increased participation in 2017, partly due to the eligibility of non-electrically heated homes for MSHP installations and the removal of pre-approval processes. However, energy savings were lower than expected due to decreased unitary savings for MSHPs and ASHPs and higher free-ridership levels for new participants. Peak demand savings were higher than tracked due to adjustments for installations in non-electrically heated homes.

er than those tracked by ENS due to the adjustment made for participants who installed an MSHP or ASHP in non-electrically heated households, for which ENS had tracked 0.0 MW in peak demand savings. Project No. 6138 18 Evaluation of 2017 D...

AI summary In 2017, the Efficient Product Installation program saw 13,672 households participate, with 377,875 efficient products installed. The program achieved 17.877 GWh in net energy savings and 2.129 MW in net peak demand savings. Evaluated savings were slightly lower than tracked savings, primarily due to a decrease in the installation rate of smart power controllers.

r 2016 levels. Overall, the evaluated net energy savings are 8 percent lower than the tracked values due to downward adjustments to both Mail-in and Instant Rebates savings (3% and 11% respectively). For Mail-in, due to efforts made by ENS...

AI summary The evaluated net energy savings for 2017 are 8% lower than tracked values, mainly due to adjustments in Mail-in and Instant Rebates. Net peak demand savings are 34% lower than tracked values, with significant reductions attributed to a single HVAC project and changes in the peak coincidence factor for indoor lighting products.

Efficiency Nova Scotia Executive Summary The level of free-ridership was established for most program components by conducting surveys or in- depth interviews. These surveys or interviews included a set of questions that were used to estab...

AI summary The free-ridership levels for most Efficiency Nova Scotia program components were determined through surveys and interviews. A weighted average of savings was used to estimate free-ridership, and cross-influence from previous participation was considered. For some components, such as Building Optimization and Energy Management Information Systems, free-ridership was considered nil due to limited market penetration and knowledge.

ponents and the limited knowledge of participants on recommissioning, or energy management information systems prior to participation explain why the level of free-ridership was considered to be nil. The impact evaluation included an asses...

AI summary The document discusses the evaluation of free-ridership and internal spillover levels for energy efficiency programs, using surveys and interviews with participants. These factors were used to calculate the net-to-gross ratio (NTGR) for each program component, with the exception of Instant Savings and New Home Construction, which involved market effects.

Mid-course Evaluated Variance Planned Savings Results DSM Program Program Component GWh MW GWh MW GWh MW (%) (%) Residential Residential Appliance Retirement 2.6 0.3 3.094 0.443 18% 50% Efficient Product Rebates Instant Savings 20.0 3.3 19...

AI summary The document evaluates the performance of various demand-side management (DSM) programs, comparing planned savings against actual results. Residential programs like appliance retirement and efficient product rebates show mixed outcomes, with some programs underperforming while others slightly exceed expectations. BNI programs also show positive variance in energy savings.

erformed. As for partners, most were satisfied with their interactions with ENS and value their involvement in the program. Table 11 below presents satisfaction scores obtained in the 2017 evaluation. Participants gave all program componen...

AI summary The 2017 evaluation of Efficiency Nova Scotia's DSM programs shows high participant satisfaction across most components, with the exception of New Home Construction. The merger of two programs into Efficient Product Installation was well-received, but partner satisfaction with Instant Savings declined due to frustrations with a price floor for LED products.

y indicated that ENS marketing was the most common way for participants to find out about Instant Savings, primarily through in-store promotions, followed by TV ads, the ENS website and word-of-mouth. This year, the Evaluator reviewed the...

AI summary The Instant Savings program used ENS marketing, primarily in-store promotions, to reach participants. The Evaluator reviewed unitary savings values for LED lamps, washers, and refrigerators, noting a decline in free-ridership and spillover effects. The program achieved 19.543 GWh in net energy savings and 2.558 MW in peak demand savings in 2017, avoiding 12,764 tonnes of CO2 eq annually.

ements were also praised. Although EAs viewed the tiered incentive approach as an effective one for encouraging participants to implement more measures, they found the rebate calculation challenging. Project No. 6138 41 Evaluation of 2017...

AI summary The evaluation of the 2017 DSM programs found that the tiered incentive approach was effective, but rebate calculations were challenging. Home Energy Assessment saw increased electrical savings due to more participants installing space-heating appliances. Free-ridership was 25%, slightly lower than 28% in 2016, and internal spillover was 1%, lower than previous years. The NTGR was calculated at 0.76.

do not generate high savings compared to other spillover measures observed in previous years. By applying these free-ridership and internal spillover levels, the Evaluator established the NTGR at 0.76 Home Energy Assessment aimed to achiev...

AI summary The Home Energy Assessment program achieved higher energy and peak demand savings than expected, with 7.007 GWh in net energy savings and 1.981 MW in net peak demand savings. The Evaluator recommends simplifying the incentive structure to improve clarity for EAs and participants.

nges, the Evaluator recommends continuing this effort while trying to make this structure as simple and straightforward as possible for both EAs and participants. HEA-R2. Conduct a billing analysis to review the overestimation ratio. The E...

AI summary The document discusses the evaluation of the 2017 DSM programs, highlighting changes to the Green Heat program in August 2017, including its relaunch as a mail-in rebate program and the elimination of the pre-approval process. It also mentions the need for a billing analysis to address overestimation ratios in the HEA program.

t since 2011. A total of 420 participants completed projects and generated electrical savings under New Home Construction in 2017. Six participants completed a project through the Passive House pilot. The participants and builders intervie...

AI summary The New Home Construction program and Passive House pilot in 2017 had moderate participant satisfaction, with participants and builders citing low rebates and complicated processes as main issues. Free-ridership was 29% for New Home Construction and 61% for the Passive House pilot, with market effects at 35% for New Home Construction, showing an increase from 2016.

hip at 61 percent. Based on the free-ridership and market effects levels established, the NTGRs for New Home Construction and the Passive House pilot were established at 1.06 and 0.39 respectively. Project No. 6138 47 Evaluation of 2017 DS...

AI summary The evaluation of the 2017 DSM programs by Efficiency Nova Scotia found that New Home Construction and the Passive House pilot achieved significant energy and peak demand savings, contributing to GHG emission reductions. The Evaluator recommends optimizing the program component based on the results.

straightforward participation process. Market data should also be collected to obtain insights about market forces that may be influencing the decline in NHC participation. NHC-R2. Conduct a billing analysis to review the overestimation ra...

AI summary The text discusses the need for a billing analysis to assess the overestimation ratio in the New Home Construction (NHC) program, as well as participation trends in business energy rebate programs. It highlights the importance of market data collection and the increase in rebate product sales in 2017.

Efficiency Nova Scotia Executive Summary The level of satisfaction with Mail-in remains high for 2017, with participants rating it on average 9.0 on a 10-point scale. Participants were asked about specific aspects of Mail-in, namely intera...

AI summary Participant satisfaction with Mail-in remains high, with an average score of 9.0. Satisfaction with Instant Rebates is lower, averaging 6.5. Small adjustments were made to tracked energy savings, and free-ridership levels for Mail-in and Instant Rebates were measured at 14% and 27%, respectively.

d free-ridership levels for four LED product categories. Those products have a free-ridership level between 32 and 38 percent, while the overall free-ridership level for Instant Rebates is 27 percent. Business Energy Rebates aimed to achie...

AI summary The Business Energy Rebates program achieved significant energy savings in 2017, exceeding its targets. Instant Rebates contributed notably, with high participation and increased unitary savings. The program also helped avoid 30,423 tonnes of CO2 eq in annual GHG emissions. The Evaluator recommends improvements based on the evaluation results.

Executive Summary Table 21: Recommendations for Business Energy Rebates No. Recommendations BER-R1. Continue to monitor the market evolution of LED lamps and adapt the BER offer when needed. The 2017 evaluation served to confirm solid mark...

AI summary The document recommends that ENS continue to monitor the evolution of the LED market, particularly as market barriers decrease and adoption increases. It suggests that ENS should remove certain product categories from the Business Energy Rebate program when market transformation reaches a high level, based on indicators such as retail prices and market share.

uator recommends that ENS identify BNI sectors that are currently underrepresented in the Mail-in service and that could potentially generate significant savings. BER-R3. Continue and improve the tracking of BER Instant Rebates participant...

AI summary The document discusses recommendations for improving the tracking of energy efficiency programs, particularly the BER Instant Rebates, and highlights the performance of ENS's Custom services in 2017, noting that Retrofit was the most successful, while New Construction and Building Optimization had low participation.

Moreover, one consultant and one participant, both with experience in using other efficiency programs elsewhere in Canada, described the ENS Retrofit service as exceptional in comparison to its peers. For Retrofit, the Evaluator’s review o...

AI summary The ENS Retrofit service was praised for its quality, but the evaluation found issues with M&V procedures and data documentation, leading to adjustments in energy savings calculations. Discrepancies in simulation models and errors in savings calculations were identified in New Construction and Building Optimization projects. Free-ridership was not measured for Retrofit in 2017 and was confirmed nil for Building Optimization.

ipant-specific baseline used by ENS results in gross savings being net of free-ridership. Free-ridership for Building Optimization was confirmed in 2017 to be nil through interviews with participants. Collectively, Retrofit, New Constructi...

AI summary The evaluation of 2017 DSM programs by Efficiency Nova Scotia found that the participant-specific baseline used resulted in net savings after accounting for free-ridership, which was confirmed as nil for Building Optimization. The programs achieved significant energy and peak demand savings, equivalent to 13,719 tonnes of CO2 eq in avoided emissions.

make his or her building function normally and the real energy-saving opportunities for improving energy performance above a minimum level. Better objective-setting would allow Project No. 6138 52 Evaluation of 2017 DSM Programs Efficiency...

AI summary The evaluation of 2017 DSM programs highlights the need for better documentation of basecase conditions for Building Optimization projects and notes that Energy Management Information Systems (EMIS) projects showed limited progress in generating savings compared to previous years.

m of the auditor and usefulness of the audit. The other few participants who were less satisfied indicated dissatisfaction with the amount of the incentive (n=2) and the energy savings achieved (n=2). The Evaluator reviewed ENS’s gross sav...

AI summary The evaluation of ENS's energy efficiency programs found that most participants were satisfied with the audit process, though some were dissatisfied with incentive amounts and energy savings. The Evaluator reviewed savings estimates and found discrepancies due to site conditions, leading to adjustment ratios for energy and peak demand savings. Free-ridership was found at 7% for the audit path and 13% for the DIY path, with no spillover effects.

statcan.gc.ca/pub/64-001-x/64-001-x2015012-eng.pdf (Last accessed January 15, 2018) STATISTICS CANADA. Catalogue Table 026-0002 Building permits, dwelling units by type of dwelling and area, 2016, available at http://www5.statcan.gc.ca/can...

AI summary This section discusses the calculation of margins of error for various programs, including the Efficient Product Installation, Business Energy Rebates Mail-in, and Green Heat programs. It outlines the methodology used to determine margins of error for surveys, installation rates, adjustment ratios, free-ridership levels, and energy consumption measurements.

Efficiency Nova Scotia Executive Summary – Appendix Report APPENDIX III NTGR CALCULATIONS This appendix presents some examples of how the Evaluator calculated free-ridership, internal spillover and net-to-gross ratio values in the 2017 eva...

AI summary This appendix provides examples of how the Evaluator calculated free-ridership, internal spillover, and net-to-gross ratio values in the 2017 evaluation of Efficiency Nova Scotia's Green Heat and Small Business Energy Solutions programs. It references an Excel file and evaluation reports for further details.

- 0 - 0 - 5 - Efficiency Trade Network PA3 Score (10 − FR4) × 10% - 10% - 50% - 0% Project No. 6138 22 Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive Summary – Appendix Report

AI summary The document presents an evaluation of the 2017 Demand Side Management (DSM) programs by Efficiency Nova Scotia, including a PA3 score calculation and a reference to Project No. 6138.

PA2 = EMPTY IF PA4 < MEAN (PA1; PA2; PA3): MEAN (PA1; PA2; PA3; PA4) Free-ridership - 3% - 38% - 20% OTHERWISE MEAN (PA1; PA2; PA3) CI2/4a. [The company’s previous participation in an ENS program]/[The energy efficiency promotional materia...

AI summary The text presents a calculation method for free-ridership and includes survey questions related to the impact of ENS programs and promotional materials on company decisions to install energy-efficient upgrades and conduct energy audits. The questions assess whether the company's participation in ENS programs influenced their evaluation of energy efficiency options.

- 0 - 0 - 10 - Efficiency Trade Network PA3 Score (10 − FR4) × 10% - 10% - 100% - 0% Project No. 6138 24 Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive Summary – Appendix Report FR3b.[IF TOTAL LIGHTING QTY >1] If the prog...

AI summary The text presents a section from an evaluation of 2017 DSM (Demand Side Management) programs by Efficiency Nova Scotia. It includes a scoring mechanism (PA3 Score) and a question (FR3b) assessing the likelihood that participants would have installed the same quantity of lighting products without the program's rebate or financing.

FR3c. [ASK IF FR2 a or b ≥5] If the program rebate or financing had not been offered, what is the likelihood that you would have postponed installing the lighting FR3c = (10 − Answer) x 10% NA 0 100% 5 50% upgrades by at least one year? IF...

AI summary The text outlines a method for assessing free-ridership and program effectiveness through various performance assessments (PA1, PA2, PA3, PA4) and consistency tests. It includes a scale for evaluating the likelihood of delayed installation of lighting upgrades and calculates scores based on responses.

- 3% - 100% - 18% OTHERWISE MEAN (PA1; PA2; PA3) CI2/4a. [The company’s previous participation in an ENS program]/[The energy efficiency promotional materials distributed by ENS] was a major factor in your 1 0 1 decision to install energy...

AI summary The text presents a series of questions to assess the impact of ENS programs and promotional materials on a company's decision-making regarding energy efficiency upgrades. It asks respondents to agree or disagree with statements about whether ENS programs influenced their decisions, prompted energy audits, or affected cost-effectiveness evaluations.

ency Nova Scotia Executive Summary – Appendix Report Small Business Energy Solutions - DYI Path HVAC Low Free-ridership High Free-ridership Average Free-ridership Free-ridership algorithm Participant Participant Participant Calculation Cal...

AI summary The text discusses a free-ridership algorithm (FR2ab) used to assess whether participants in the Small Business Energy Solutions - DYI Path HVAC program would have paid for HVAC upgrades without receiving rebates or financing from ENS. The algorithm calculates free-ridership based on responses to a question about willingness to pay.

e Network PA3 Score (10 − FR4) × 10% - 0% - 70% - 0% Project No. 6138 26 Evaluation of 2017 DSM Programs Efficiency Nova Scotia Executive Summary – Appendix Report FR3c. [ASK IF FR2 a or b ≥5] If the program rebate or financing had not bee...

AI summary The text presents a scoring framework for evaluating the impact of energy efficiency programs, specifically focusing on the likelihood of delayed HVAC upgrades if rebates or financing were not available. It includes a scoring calculation for FR3c based on a scale from 0 to 10.

2 80% 9 10% IF DK OR REF: FR3b = EMPTY HVAC upgrades by at least one year? (Scale 0 to 10) PA4 Score FR3c - - - 80% - 10% IF ABS(FR2ab − FR3a) ≥ 50% : Inconsistency Test - PA1 - 20% - 55% - 50% PA1 = MIN(FR2ab; FR3a) IF FR1a = 100% AND FR2...

AI summary The text discusses performance assessments (PA1 to PA4) and free-ridership (FR) metrics, including inconsistency tests and calculation methods. It outlines conditions for determining scores and how they are evaluated based on percentages and thresholds.

OTHERWISE MEAN (PA1; PA2; PA3) CI2/4a. [The company’s previous participation in an ENS program]/[The energy efficiency promotional materials distributed by ENS] was a major factor in your 1 0 1 decision to install energy efficient upgrades...

AI summary The text presents survey questions assessing the impact of ENS programs and promotional materials on a company's decision-making regarding energy efficiency upgrades. The questions focus on whether ENS initiatives influenced the company's decisions to install upgrades, conduct energy audits, and consider cost-effectiveness of energy-efficient options.

n 0 each upgrade BER program Additional Upgrade Savings SO4 - 0 SO3. Did your experience with this program influence your decision to implement these additional energy efficiency measures on your own? SO3 = Answer x 10% 0 0% (Scale 0 to 10...

AI summary The text discusses the calculation of spillover savings and free-ridership in an energy efficiency program, with the NTGR (Net Total Gross Reduction) calculated as 88%. It includes survey data on participant experiences and their influence on additional energy efficiency measures.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text defines key terms related to energy efficiency program evaluations, including gross and net savings, free-ridership, induced consumption, and interactive effects. It outlines the evaluation process and the parameters used to calculate energy savings.

y measure introduced by a program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to programs throug...

AI summary The text defines several key terms related to energy efficiency programs, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These terms are used to evaluate the effectiveness and impact of energy efficiency initiatives.

Errors arising during the course of all survey activities other than sampling. Non-sampling error Non-sampling errors tend to lead to bias in measurements and are very difficult to quantify. Peak coincidence The percentage of measure deman...

AI summary The text discusses various statistical and measurement concepts in energy efficiency programs, including non-sampling errors, precision, sample size, and tracked savings. It also defines terms such as peak coincidence factor, unitary savings, and secondary market impacts, which are relevant to energy efficiency tracking and evaluation.

fficiency codes and standards introduced in Nova Scotia. To carry out this assignment, Econoler collaborated with Research Into Action, Equilibrium Engineering and Corporate Research Associates (CRA). Econoler served as team leader in the...

AI summary Econoler led a 2017 evaluation of energy efficiency programs in Nova Scotia, collaborating with multiple organizations. The evaluation included impact assessments, process evaluations, and data collection, resulting in reports on energy savings and GHG emissions. Table 6 outlines the programs assessed and the types of evaluations conducted.

Life Review (n=1) (n=3) Analysis Report The Evaluator reviewed the program component documentation and then met with the program manager in August 2017 to verify the implementation status of the previous evaluation recommendations and lear...

AI summary The Evaluator reviewed program documentation and met with the program manager to assess the implementation of previous evaluation recommendations and changes made in 2017. A data collection plan was developed based on this review, aiming for a 10% margin of error at a 90% confidence level.

Report of error indicates the relative level of precision4 of the measurement, while its associated confidence level indicates the probability of a measurement falling within this margin of error.5 As part of this evaluation, the Evaluator...

AI summary The evaluation process includes calculating margins of error for survey samples, free-ridership levels, and energy consumption metering. An interview with the Halifax Regional School Board and on-site visits to a recycling facility were conducted to assess the Appliance Retirement Pilot program. Unitary savings calculations were based on metering data and prior evaluations, with no new survey conducted in 2017.

able, no participant survey was conducted in 2017 and the 2016 survey results were used to establish the free-ridership level, secondary market impacts and induced consumption and the associated NTGR. For HomeWarming, the data collected in...

AI summary The 2017 evaluation of the Residential Efficient Product Rebates Program by Efficiency Nova Scotia (ENS) involved a literature review of technical reference manuals and evaluation reports to assess effective useful life (EUL) values used for calculating energy savings. Free-ridership levels and energy consumption differences were analyzed using data from surveys and appliance retirement programs.

e energy savings, the Evaluator conducted a thorough literature review by consulting technical reference manuals (TRMs), evaluation reports and relevant studies. Comparison with 2016 Evaluation Scope No market evaluation took place for ARe...

AI summary This section discusses the impact evaluation of the Appliance Retirement (ARET) program in 2017, including the metering review and calculation of energy savings. The Evaluator conducted a sampling plan, site visits, and data analysis to assess the effectiveness of the program.

Efficiency Nova Scotia 2017 DSM Evaluation Report Average Energy Consumption of Replacement Room Air Conditioners The Evaluator assumes that the replacement units meet the current minimum level of energy efficiency required to meet ENERGY...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia discusses the average energy consumption of replacement room air conditioners, assuming they meet ENERGY STAR efficiency levels. It calculates annual energy consumption and net unitary savings, considering factors like free-ridership and secondary market impacts.

usage conditions, the Evaluator applied a factor of 1.00 since it is assumed that all appliances retired so far through the pilot were rarely used since they were considered at as surplus appliances. According to the HRSB representative in...

AI summary The evaluation of the appliance retirement pilot program applied a part-use factor of 1.00 to retired appliances, assuming they were rarely used. Interviews with HRSB representatives confirmed that retired appliances were found plugged in, supporting the use of this factor. The report also discusses free-ridership assessment and the roles of HRSB and school principals in the pilot's implementation.

- Gross Peak Demand Savings – at the Generator 0.001 0.000 0.000 0.006 0.043 0.030 0.740 (MW) Appliance Retirement 53 Residential Efficient Product Rebates Program Efficiency Nova Scotia 2017 DSM Evaluation Report 3.14 Internal Spillover F...

AI summary The document discusses internal spillover effects from the Appliance Retirement (ARET) program and the methodology used to calculate the net-to-gross ratio (NTGR). It references a 1% spillover rate from the 2016 evaluation and a 0% rate for the ARet Pilot based on HRSB interviews. The NTGR calculation method involves weighted averages of savings scenarios.

cipants affected, but also the difference in savings associated with each scenario. This methodology was followed to establish the net unitary savings, as described in previous sections of the report. To obtain an equivalent NTGR that take...

AI summary The text discusses a methodology for calculating net unitary savings and equivalent NTGR (Net to Gross Ratio) by considering both unitary savings and spillover effects. Table 41 provides data related to appliance retirement and residential efficient product rebates under the Efficiency Nova Scotia 2017 DSM Evaluation Report.

Efficiency Nova Scotia 2017 DSM Evaluation Report 4.2 Follow-up on 2016 Evaluation Report Recommendations Instant Savings was evaluated in 2016 and improvement recommendations were made by the Evaluator. Table 45 below provides a brief sum...

AI summary The 2017 DSM Evaluation Report discusses the follow-up on recommendations from the 2016 evaluation of the Instant Savings program. Key recommendations included monitoring market indicators and specific appliance markets, both of which have been implemented.

300 000 200 000 5 100 000 0 0 2011 2012 2013 2014 2015 2016 2017 2011 2012 2013 2014 2015 2016 2017 CFLs LEDs Other Products CFLs LEDs Other Products Instant Savings 62 Residential Efficient Product Rebates Program Efficiency Nova Scotia 2...

AI summary The 2017 Instant Savings evaluation methodology involved data collection activities such as program documentation review, interviews with the program manager, unitary savings review, effective useful life review, and participant surveys. These activities were used to evaluate the program's performance and outcomes.

etailers Participants n=8 n = 100 Analysis Report The Evaluator reviewed the program component documentation and then met the program manager in August 2017 to verify the implementation status of previous evaluation recommendations and lea...

AI summary The Evaluator reviewed program documentation and met with the program manager to assess implementation status and changes. A data-collection plan was developed, and quantitative evaluation activities were conducted with a 10% margin of error at a 90% confidence level. Margins of error were calculated for survey samples and free-ridership rates and are presented in the report.

nted alongside the results in the applicable sections. Examples of calculations used to establish margins of error can be found in Appendix II of the 2017 DSM Programs Evaluation Executive Summary. Instant Savings 63 Residential Efficient...

AI summary The document discusses evaluations of the 2017 DSM Programs, including interviews with retailers and a participant survey conducted in Halifax to assess the Instant Savings program and its impact on ENERGY STAR certified LED lamps.

estionnaire and complete participant survey results are presented in Appendices V and VI respectively. The intercept participant survey generated feedback on the following aspects of Instant Savings: › LED lamp purchases and expected uses...

AI summary The document presents survey results and evaluations related to the Instant Savings program, focusing on participant feedback, unitary savings reviews for LED lamps and other products, and the Effective Useful Life review. The Evaluator used literature reviews, engineering calculations, and technical references to assess savings values.

acking sheet. Technical reference manuals and public evaluation reports of jurisdictions similar to ENS were consulted with focus on the most recent and accurate sources. Effective Useful Life Review As part of the 2017 evaluation, to exam...

AI summary The 2017 evaluation of the Instant Savings program included interviews with retailers and a review of energy savings calculations using Effective Useful Life (EUL) values. The evaluation scope was adjusted compared to 2016 due to changes in campaign timing and product analysis.

e-Dehumidifier ENERGY STAR Certified Bathroom Fans 0.278 Calculated by the Evaluator (without a light) Interactive Effects In a home, interactive effects occur when the implementation of energy efficiency measures has an impact on the ener...

AI summary The document discusses the interactive effects of energy efficiency measures, specifically focusing on how replacing incandescent lighting with LED lamps affects heating and cooling loads. It also outlines ENS's approach to calculating these effects, including the use of different factors for energy savings and peak demand savings.

ter Line Loss Factor 1.100 1.100 1.100 1.100 1.100 1.100 1.100 1.100 1.100 1.100 1.100 - Gross Peak Demand Savings – 0.000 0.000 -0.282 0.000 0.005 0.002 0.028 0.019 0.013 0.043 0.002 2.993 at the Generator Instant Savings 109 Residential...

AI summary The 2017 DSM Evaluation Report discusses the Net-to-gross Ratio (NTGR) for the Residential Efficient Product Rebates Program, focusing on free-ridership and spillover effects, particularly for LED lamps. The report uses a self-reporting approach from participants to assess free-ridership levels for LED lamps, which account for 92% of tracked gross savings.

ogram component. The free-ridership level was assessed using a self-reporting approach which involved asking participants who had purchased LED lamps a series of questions during the intercept survey. The free-ridership level is calculated...

AI summary The free-ridership level for LED lamps sold through the Instant Savings program in 2017 was 32%, calculated using a self-reporting approach and an algorithm that considers factors like discount awareness and purchase decisions. This level is similar to 2016, indicating market stabilization. The survey also showed improved program visibility, with fewer participants unaware of the rebate. Spillover effects may have influenced sales outside of campaign periods due to increased awareness of energy-efficient products.

ncandescent technology efficiency. 8.2 Measured Free-ridership The Evaluator looked at the free-ridership level to gain insights into the evolution of the LED residential market in Nova Scotia. Given the reduction in market barriers to LED...

AI summary The evaluation discusses free-ridership levels in the LED residential market in Nova Scotia, noting a 32% free-ridership rate for LED lamps sold through the Instant Savings program in 2017, similar to 2016 and lower than 2015. The analysis suggests that despite high free-ridership, incentives should continue due to the significant number of participants who would not have purchased LED lamps without the program.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report With this in mind, approximately what percentage of the number of ENERGY STAR LED bulbs you sold outside of the two campaign periods can you directly or indirectly attribute to E...

AI summary The text asks Efficiency Nova Scotia to estimate the percentage of ENERGY STAR LED bulb sales outside of specific campaign periods that can be attributed to their programs and promotions. It also requests an explanation and includes a satisfaction rating question for the Instant Savings program.

lle construction au marché résidentiel, Final Report presented to Hydro-Québec, February 9, 2009. 9 Ontario Power Authority (OPA), 2011 Prescriptive Measures and Assumptions Version 1, March 2011. Project No. 6138 35 Residential Efficient...

AI summary The text discusses the algorithm for calculating free-ridership levels in the Residential Efficient Product Rebates Program, specifically focusing on participants who were unaware of discounts before paying at the cash register. The algorithm uses survey responses to assess program awareness and attribution levels.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report IS: INTERNAL SPILLOVER METHODOLOGY The retailer interviews were used to estimate the level of spillover by measuring the market effects attributable to the program for LED lamps....

AI summary The document discusses the methodology used to estimate internal spillover effects from the Instant Savings program, focusing on LED lamp sales during spring and fall campaigns. Retailers were interviewed to assess program attribution for eligible product sales outside the campaigns, allowing for the estimation of market effects.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text defines various energy efficiency program evaluation terms, including gross and net savings, first-year savings, free-ridership, induced consumption, and interactive effects. It outlines the evaluation process and the parameters used to calculate energy savings.

y measure introduced by a program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to programs throug...

AI summary The text defines several key metrics used in energy efficiency programs, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These metrics help assess program effectiveness, energy savings, and market impacts.

Errors arising during the course of all survey activities other than sampling. Non-sampling error Non-sampling errors tend to lead to bias in measurements and are very difficult to quantify. Peak coincidence The percentage of measure deman...

AI summary The text discusses various concepts related to energy efficiency and measurement, including non-sampling errors, peak coincidence, precision, random sampling error, sample size, secondary market impacts, tracked savings, and unitary savings. These concepts are relevant to evaluating the effectiveness of energy efficiency programs.

LIST OF TABLES Table 1: Overall Participation and Savings for Existing Residential ..................................................... xiii Table 2: HEA Net-to-gross Ratio ....................................................................

AI summary The text presents a list of tables from a regulatory proceeding, focusing on energy efficiency programs, savings tracking, and evaluation metrics. It includes tables related to participation, net-to-gross ratios, energy savings comparisons, program implementation status, and evaluation types.

2017 Free-ridership Level 25% Spillover Level 1% Net-to-gross Ratio 0.76 Another spillover effect associated with unconverted D assessment savings was also considered in the net energy and peak demand savings calculations. This spillover c...

AI summary The 2017 DSM Evaluation Report discusses free-ridership and spillover effects in the Home Energy Assessment (HEA) program. It calculates net energy and peak demand savings by accounting for unconverted D assessment participants and uses the 2016 evaluation data. Net savings for 2017 were 7.007 GWh in energy and 1.981 MW in peak demand.

Nova Scotia 2017 DSM Evaluation Report Table 3: Comparison of HEA Tracked and Evaluated Savings at the Generator Initial Gross Adjusted Gross NTGR Net Savings Savings Savings Energy Savings Tracked Savings from ENS 8.945 GWh 8.298 GWh 0.78...

AI summary The 2017 DSM Evaluation Report compares tracked and evaluated energy and peak demand savings from the Home Energy Assessment (HEA) program. Evaluated net savings are 2% lower than tracked savings due to a decrease in the NTGR value. HEA received positive feedback and exceeded planned savings, with specific recommendations for improvement provided in the report.

submit the required documentation to receive incentives, and financing has been discontinued. That said, ENS is working with Nova Scotia Power to cross-promote the on-bill financing option they offer. Other changes under the new path relat...

AI summary The Green Heat program underwent changes, including the discontinuation of financing and adjustments to heat pump incentives. The program achieved significant energy and peak demand savings in 2017, with evaluations based on documentation reviews, interviews, and participant surveys.

are significantly lower than those tracked by ENS. This is partially because participants often have other heating systems (electrical or non-electrical) that also contribute to the home heating load. The net-to-gross ratio (NTGR) values l...

AI summary The net-to-gross ratio (NTGR) values for the Green Heat program are influenced by free-ridership levels, with new path participants showing higher free-ridership due to simplified application processes. The NTGR for heat pump water heaters is set at 1.00 due to their recent introduction and cost.

Efficiency Nova Scotia 2017 DSM Evaluation Report Table 5: Comparison of Green Heat Tracked and Evaluated Savings at the Generator Initial Gross Savings Adjusted Gross Savings NTGR Net Savings Energy Savings Tracked Savings from ENS 8.510...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia shows that evaluated gross energy savings are 41% lower than tracked savings, primarily due to reduced unitary savings for MSHPs and ASHPs. Evaluated net energy savings are 42% lower, attributed to higher free-ridership levels, especially for new path participants with electric heating who installed MSHPs.

unable to provide an indication of MSHP market adoption on the theoretical curve of technology diffusion as further data is required to fully assess the transformation of this market in Nova Scotia.4 Green Heat Recommendations The Evaluato...

AI summary The evaluation of the Green Heat program found that while participation increased, energy savings from MSHP and ASHP installations were lower than expected. Recommendations include re-evaluating ASHP savings calculations with a larger sample size to improve accuracy.

s eligible for a rebate, ENS should develop a list of eligible equipment, similar to the list established for MSHPs. Contractors could be asked to develop and update this list. Efficient Product Installation In January 2017, the former Ren...

AI summary The EPI program component, formed in 2017 by merging former programs, provides free installation of energy-efficient products to Nova Scotia residents, with a focus on low-income households. In 2017, EPI achieved 17.877 GWh in net energy savings and 2.129 MW in net peak demand savings, though participation and product installations decreased compared to the previous year.

Efficiency Nova Scotia 2017 DSM Evaluation Report The average number of products per participant has increased overall, partially because of the higher proportion of participants living in single-family homes in 2017, which typically have...

AI summary The 2017 DSM Evaluation Report highlights that the average number of products per participant increased, partly due to more single-family home participants. However, A-type lamp replacements decreased significantly, impacting EPI's gross savings. EPI received high participant satisfaction ratings, with positive feedback on installer interactions and overall service quality.

the NTGR was separately established for non- low-income owners and tenants, because the Evaluator expected different results for each group. For low-income participants, the NTGR was assumed to be 1. Existing Residential Program xxvii Exis...

AI summary The document discusses the calculation of the Net to Gross Ratio (NTGR) for residential energy efficiency programs. It outlines different NTGR values for low-income and non-low-income participants, with the overall weighted average NTGR calculated at 0.92. Free-ridership and internal spillover levels are also analyzed for LED lamps and other products.

interviews and surveys with participants and partners, as well as analyzed and reported on the data collected from these activities. Throughout this report, this team is referred to as the Evaluator. For each program, the Evaluator prepare...

AI summary The document discusses the evaluation of demand-side management (DSM) programs conducted by Efficiency Nova Scotia (ENS) in 2017. It outlines the evaluation process, including interviews, surveys, and data analysis, and refers to Table 8, which lists the types of evaluations conducted for each program component in 2016 and 2017.

Residential Efficient Appliance Retirement C C R Product Rebates Program Instant Savings C C R R Home Energy Assessment C C C C Existing Residential Green Heat C C C C C Program Efficient Product Installation C C C C New Residential Progra...

AI summary The document outlines various residential and business energy efficiency programs, including appliance rebates, home energy assessments, and custom incentive programs. It highlights initiatives such as the Efficient Product Installation and the Strategic Energy Management program, which are being evaluated for their effectiveness.

n (EAP) to help homeowners who may struggle with how to move forward with upgrades. The EAP includes brief information about the energy assessment, as well as the estimated costs, savings and payback. Homeowners then have 12 months to comp...

AI summary The Home Energy Assessment (HEA) program provides homeowners with guidance on energy upgrades, including pre- and post-retrofit assessments. The program aimed for 5.4 GWh in net energy savings and 1.3 MW in net peak demand savings in 2017. Follow-up actions from a 2016 evaluation report have been implemented, including assigning staff to answer questions, using unitary savings for DHW measures, and maintaining follow-ups with EAs.

0.824 1.216 1.041 2011 2012 2013 2014 2015 2016 2017 2011 2012 2013 2014 2015 2016 2017 Home Energy Assessment 6 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation Report 2 HEA EVALUATION METHODOLOGY This section prese...

AI summary The 2017 HEA evaluation methodology involved reviewing program documentation, meeting with the program manager, and preparing data collection instruments based on the information obtained.

ed. Once the meeting with the program manager and the program component documentation review were completed, specific evaluation activities were undertaken, as described in the following subsections. For evaluation activities that yield qu...

AI summary The evaluation of the Existing Residential Program by Efficiency Nova Scotia aims to achieve a 10% margin of error at a 90% confidence level for quantitative results based on sampling. This ensures that measurements are reproducible, though it does not account for accuracy or non-sampling errors such as data entry or response biases.

Efficiency Nova Scotia 2017 DSM Evaluation Report As part of this evaluation, the Evaluator calculated the margin of error for survey samples, free-ridership levels, installation rates and savings adjustment ratios where applicable. The ma...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia includes an analysis of survey margins of error, interviews with Energy Advisors, and a participant survey to gather feedback on the Home Energy Assessment program. Topics covered include participant motivations, satisfaction, verification of upgrades, free-ridership, and program improvements.

s-influence from ENS promotion and other program components) › Internal spillover › Barriers to the implementation of recommended upgrades › Recommendations for improvements › Demographics The total number of HEA participants was 931 for t...

AI summary The 2017 DSM Evaluation Report discusses the Home Energy Assessment (HEA) program component, including participant perspectives, survey results, and an evaluation of the Effective Useful Life (EUL) values used by ENS. A sample of 70 participants was surveyed to assess satisfaction, barriers, and program influence.

t is made to account for the savings separately since these upgrades are not modelled using HOT2000. Finally, the percentage of electric heating from the D assessment is applied to the overall result. This method was used in the tracking s...

AI summary The document discusses a method for calculating energy savings in residential programs, using the Home Energy Assessment (HEA) and HOT2000 software. An overestimation ratio of 16% is applied to adjust for overestimations in space heating consumption. The method was used in the 2017 evaluation and was developed by Efficiency Nova Scotia (ENS) and the Evaluator.

ly, the NTGR represents the positive or negative impact of effects on the gross savings. For HEA, two effects are considered, namely free-ridership and internal spillover. 4.2.1 Free-ridership Free-ridership can occur when participants wou...

AI summary The free-ridership level for the Home Energy Assessment program was assessed at 25%, based on a self-report telephone survey. This level is slightly lower than previous years, indicating a decrease in the proportion of participants who would have implemented energy efficiency measures without the program.

n 2012 and 31% in 2011). Table 15: Average Free-ridership Level Average Free-ridership Level Sample Size Population Size Margin of Error 25% 61 931 5.2% 4.2.2 Internal Spillover The participant survey was also used to assess internal spill...

AI summary The text discusses free-ridership and internal spillover levels in an energy efficiency program. The average free-ridership level was 25% with a 5.2% margin of error, while internal spillover was 1% with a 4.1% margin of error. The NTGR was calculated as 0.76 using these factors.

Efficiency Nova Scotia 2017 DSM Evaluation Report All 2016 Green Heat recommendations have been implemented by ENS, except for Recommendation 2016 GH-R2 which still requires some effort. To implement that recommendation, ENS has developed...

AI summary Efficiency Nova Scotia (ENS) has implemented most 2016 Green Heat recommendations, but Recommendation 2016 GH-R2 still needs work. ENS has used digital marketing and collaboration with contractors to raise awareness, but over one third of contractors were unaware of changes three months after implementation. Recommendation 2016 GH-R1 involves monitoring market indicators, and ENS requested a market evolution assessment, with results in Section 9.

the margin of error indicates the relative level of precision34 of the measurement, while its associated confidence level indicates the probability of a measure falling within this margin of error.35 As part of this evaluation, the Evaluat...

AI summary The document discusses the evaluation of survey results, including the calculation of margins of error and confidence levels. It outlines contractor interviews and participant surveys conducted in 2017 to assess the Green Heat and ENS programs, focusing on contractor involvement, outreach processes, and participant satisfaction.

ey. The average duration of the surveys was 17 minutes for the old path and 19 minutes for the new path. The questionnaires and complete survey results are presented in Appendices IX, X, XI and XII. 34 Precision refers to how reproducible...

AI summary The document discusses survey results from the Green Heat program, comparing the old and new paths. Surveys were conducted with participants who installed various heating equipment, and the results are detailed in appendices. The evaluation includes considerations of precision, confidence levels, and margin of error.

2017 DSM Evaluation Report The participant surveys were meant to collect feedback on the following aspects of Green Heat: › Participation motivations › Usage of the old and new equipment installed › Free-ridership (including the cross-infl...

AI summary The 2017 DSM Evaluation Report outlines participant surveys and on-site visits conducted to evaluate the Green Heat program. Surveys collected feedback on participation, equipment usage, and satisfaction, while on-site visits verified installation and gathered system usage data.

acked detailed information about the equipment (9%). Additional reasons reported by participants, though with lower frequency, included concerns about the reliability and energy savings of heat pumps. On average, participants expressed mor...

AI summary Participants in the program expressed significant concerns about equipment reliability and selecting the right equipment, with over two-thirds very concerned about reliability. Concerns about energy savings were also high, with 43% very concerned about realizing promised savings. Participants under the old path had concerns about project approval, while those using financing were worried about repaying loans.

1.903 91.027 Peak Demand Savings Tracked Peak Demand Savings (MW) 2.380 0.916 0.019 0.225 0.000 0.047 3.883 Adjustment Ratio for Peak Demand Savings 0.982 0.978 40.890 N/A N/A 1.117 - Revised Unitary Peak Demand Savings (kW) N/A N/A N/A 3....

AI summary The document discusses the Net-to-Gross Ratio (NTGR) in the context of the Green Heat program, explaining that it accounts for free-ridership effects to determine reliable energy savings attributed to the program component.

.083 0.154 0.012 0.039 2.711 Line Loss Factor 1.099 1.099 1.099 1.099 1.099 1.099 - Net Energy Savings – at the Generator (GWh) 1.889 0.243 0.091 0.170 0.013 0.043 2.980 Effective Useful Life (years) 18 18 18 18 18 25 - Net Lifetime Energy...

AI summary The text presents numerical data comparing energy and peak demand savings from the 2017 DSM Evaluation Report to those calculated in the tracking sheet, highlighting metrics such as Net Energy Savings, Net Lifetime Energy Savings, and Net Peak Demand Savings.

2017 DSM Evaluation Report Table 52 compares the energy and peak demand savings established through this evaluation to those calculated in the tracking sheet. Table 52: Comparison of Tracked and Evaluated Savings at the Generator Initial G...

AI summary The 2017 DSM Evaluation Report compares tracked and evaluated energy and peak demand savings. Evaluated gross energy savings are 41% lower than tracked savings due to lower unitary savings for MSHPs and ASHPs. Evaluated net energy savings are 42% lower due to higher free-ridership. Evaluated net peak demand savings are 18% higher due to adjustments for non-electrically heated households with MSHPs or ASHPs.

or updates to near-term code or standard requirements for heat pumps in the latest federal energy efficiency regulations and no known plans to make heat pumps mandatory. Market readiness of more efficient substitute Not applicable technolo...

AI summary The text discusses the market readiness of more efficient heat pump technologies, specifically MSHPs, and highlights the ongoing evolution of the market. It also notes the influence of the Green Heat program and the need for further data to assess market adoption. The EPI program and its evaluation are also briefly mentioned.

ed for upgrades that reduce the use of other fuels, but is only offered to low-income participants. This report only presents the evaluation results regarding the electricity ratepayer portion of EPI. EPI aimed to achieve 24.4 GWh in net e...

AI summary The Efficient Product Installation (EPI) program aimed to achieve 24.4 GWh in net electricity savings and 3.6 MW in net peak demand savings at the generator for 2017. The report follows up on recommendations from the 2016 evaluation of the Residential Program and Residential Demand Improvement (RDI) program components, indicating that several recommendations have been implemented.

-site visits with participants who received smart Implemented power controllers. In 2016, the Evaluator assessed spillover for the RDI program component. Since the RP&C and RDI program components were merged to create EPI, the Evaluator ha...

AI summary The text discusses the implementation of recommendations for the EPI program, including efforts to assess spillover effects, limit the number of smart power controllers per household, and increase on-site evaluations to establish accurate installation rates. Participation in the EPI program in 2017 was 18% lower than combined RDI and RP&C participation in 2016.

t for the EPI 2017 evaluation. Figure 19 below shows the various data collection activities carried out for the evaluation. Figure 19: Methodological Model The Evaluator first reviewed the program component documentation and then held a me...

AI summary The evaluation of the Efficient Product Installation (EPI) program in 2017 involved a review of program documentation, a meeting with the program manager, and data collection activities aimed at achieving a 10% margin of error at a 90% confidence level. The methodology focused on ensuring precision in measurement, though it did not account for non-sampling errors.

ndix XVI and Appendix XVII. All the appendices are presented in the Existing Residential Program – Appendices. The participant survey was meant to collect feedback about the following aspects of EPI. › Installation of smart power controlle...

AI summary The document discusses a survey and on-site visits conducted as part of the Efficient Product Installation (EPI) program in 2017. The survey aimed to gather participant feedback on installation satisfaction, behavioral changes, and barriers, while on-site visits validated product installations and collected technical data. Low-income participants were excluded to assess free-ridership and spillover effects.

ly assigned an NTGR of 1. All low-income participants were removed from the EPI survey sample so that the levels of free-ridership and spillover could be determined for EPI using the survey results. Efficient Product Installation 112 Exist...

AI summary The 2017 DSM Evaluation Report for Efficiency Nova Scotia discusses the unitary savings review and effective useful life review for the Efficient Product Installation (EPI) program. Literature reviews and engineering calculations were used to assess savings values and EUL for different product types.

ot water tank wrap and showerheads. According to participants, the main problem with the smart power strips was that they kept turning the television off; they also mentioned that LED bulbs flickered. Two in 10 participants started adoptin...

AI summary The document discusses participant feedback on the Efficient Product Installation (EPI) program, noting issues with smart power strips and LED bulbs, as well as increased energy-efficient behaviors among participants. It also outlines the evaluation of energy savings from the 2017 EPI program.

t installed. The resulting installation rate for this outlier was 413 percent. In the gross savings calculation, the outlier installation rate was only applied to the savings of that specific case.101 Because the smart power controllers’ r...

AI summary The text discusses the installation rates of smart power controllers, noting a low installation rate of 57% with a margin of error of 10.6%. The Evaluator sampled more participants in 2017 to improve accuracy and found that the true installation rate is likely between 46% and 68%. The low adoption rate suggests the technology is not being effectively adopted.

y Ratio 0.162 0.162 0.000 - Total Gross Peak Demand Savings – at the Meter (MW) 0.003 0.006 0.000 0.202 Line Loss Factor 1.099 1.099 1.099 - Total Gross Peak Demand Savings – at the Generator (MW) 0.003 0.007 0.000 0.222 Efficient Product...

AI summary This section discusses the Net-to-Gross Ratio (NTGR) in the context of the Efficient Product Installation (EPI) program. It explains how NTGR accounts for free-ridership and internal spillover effects, with free-ridership being assessed through a self-report approach in a telephone survey of non-low-income participants.

Efficiency Nova Scotia 2017 DSM Evaluation Report Table 79: Average Free-ridership Level by Product Category for EPI Non-low-income Participants Average Free- Population Margin Percentage of Gross Measure Category Sample Size ridership Lev...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia examines free-ridership and internal spillover levels for the Efficient Product Installation (EPI) program. The report finds that non-low-income participants had varying levels of free-riding and internal spillover, with LED lighting showing higher participation rates compared to other product categories.

As with the free-ridership levels, spillover levels were separately established for owners and tenants, because the Evaluator expected the tenants’ spillover level to be lower than that of the owners. Based on the 2017 survey results, an i...

AI summary The text discusses the establishment of internal spillover factors for non-low-income owners and tenants in the context of the Efficient Product Installation (EPI) program. The spillover factors are set at 6% for owners and 1% for tenants, based on 2017 survey results and are higher than previous years' factors. The document also mentions the NTGR calculation, which is discussed in the following section.

Efficiency Nova Scotia 2017 DSM Evaluation Report CONCLUSION The Existing Residential program is comprised of two well-established program components (HEA and Green Heat) and a new component (EPI, which has resulted from the merger of RDI...

AI summary The 2017 DSM Evaluation Report highlights that the Existing Residential program achieved its peak demand savings target but fell short of its energy savings goal. EPI was the main contributor to energy savings, while Green Heat contributed most to peak demand savings. HEA exceeded both its energy and peak demand savings targets in 2017.

Program – Appendices. Home Energy Assessment Appendix I - HEA: 2017 Recommendations Appendix II - HEA: Interview Guide with the Energy Advisors Appendix III - HEA: Participant Survey Questionnaire Appendix IV - HEA: Participant Survey Resu...

AI summary The text outlines appendices related to various energy efficiency programs, including Home Energy Assessment, Green Heat, and Efficient Product Installation, with details on surveys, methodologies, and evaluation reports from 2017.

tary Savings Values Appendix XX - EPI: Residential Lighting Metering Study Report Appendix XXI - EPI: Algorithm for Free-ridership Calculation Appendix XXII - EPI: Internal Spillover Methodology Existing Residential Program 163 160 Saint-P...

AI summary This document provides appendices related to the Efficiency Nova Scotia 2017 Demand Side Management (DSM) Evaluation, including a Residential Lighting Metering Study Report, an algorithm for free-ridership calculation, and internal spillover methodology.

. 137 EPI: PARTICIPANT SURVEY RESULTS ................................................................ 151 EPI: ON-SITE VISIT SAMPLING METHODOLOGY AND PROTOCOL................. 173 EPI: DETAILED CALCULATIONS OF UNITARY SAVINGS VALUES ........

AI summary This appendix outlines the 2017 recommendations made by the Evaluator as part of the evaluation of the Existing Residential Program by Efficiency Nova Scotia. There were no incomplete 2016 recommendations at the time of writing.

HEA: 2017 RECOMMENDATIONS This appendix summarizes all the recommendations made by the Evaluator as part of the 2017 evaluation. There were no incomplete 2016 recommendations at the time of writing. Section Recommendations 1. Review the cu...

AI summary The 2017 evaluation of the Home Energy Assessment (HEA) program recommends simplifying the incentive structure and conducting a billing analysis to address overestimation of savings due to the use of a new version of the HOT2000 software.

RVIEW GUIDE WITH THE ENERGY ADVISORS PROGRAM NAME: HOME ENERGY ASSESSMENT Interview date and time Company name Interviewee name Area code and phone number Email Introduction Thank you for taking the time to speak with me. Today, I am inter...

AI summary This document is a review guide for evaluating Efficiency Nova Scotia's Home Energy Assessment program. It outlines an interview process with program partners to gather insights on their involvement and satisfaction with the program. The evaluation is being conducted by Econoler in partnership with Corporate Research Associates.

number of homes that register for a pre-retrofit assessment (in terms of marketing, outreach activities, technical support, etc.)? Is there anything that service organizations could do? B2. What prevents homeowners from going through the p...

AI summary The text includes a series of questions aimed at evaluating the Existing Residential Program by Efficiency Nova Scotia, focusing on barriers to participation, implementation of recommended upgrades, and improvements to the Home Energy Assessment program. It also seeks insights into recent trends in energy efficiency in Nova Scotia's residential market.

Explanation: C5. What additional measures or equipment should ENS incent or support as part of the Home Energy Assessment Program or another program to increase market adoption? Why? Explanation: C6. Will it become easier or more difficult...

AI summary The text presents a series of questions from a regulatory proceeding regarding the Home Energy Assessment Program, focusing on measures to increase market adoption, challenges in promoting high-efficiency products, and broader trends in the residential market. It also includes a survey questionnaire for participants in the program.

_) 97. None 98. Don’t know 99. Refused [ASK VR5 FOR EACH ‘NO, DID NOT IMPLEMENT THE UPGRADE’ RESPONSE IN VR3a-j), OR VR2A=03 (None) or VR2B=02 (No). VR5. What is the most important reason why you did not make the [INSERT ITEM FROM VR3a-j]...

AI summary The text outlines a survey process for assessing residential energy efficiency upgrades, including reasons for not implementing upgrades and rebate preferences. It also references a program evaluation related to Efficiency Nova Scotia and mentions a free-ridership assessment.

- 32% - 37% - 41% Don’t know/Refused - 1% - 1% - 6% Multiple responses Project No. 6138 21 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report S1. Using a scale from 1 to 10 where 1 is ‘not at all sati...

AI summary The document presents survey results from the 2017 evaluation of the Existing Residential Program, specifically the Home Energy Assessment (HEA) under Efficiency Nova Scotia. It includes customer satisfaction ratings and reasons for dissatisfaction with the program.

Section Recommendations 2016 6. Increase program component advertising to households and collaborate with Recommendations contractors. Evaluation results showed fairly high free-ridership levels for products (Not fully rebated through Gree...

AI summary The evaluation of the Green Heat program highlights high free-ridership levels for rebated products, particularly heat pumps and MSHPs. The Evaluator recommends increasing advertising to households and collaborating with retailers to improve program awareness and promote higher efficiency models. Retailer training and point-of-sale materials are also suggested to enhance promotion of energy-efficient products.

amming instructions are in bracketed CAPS. › [Interviewer notes] Onscreen interviewer instructions are bracketed in italics. › [Piped value] Database inputs are in bold. Project No. 6138 50 Existing Residential Program Efficiency Nova Scot...

AI summary This document outlines the initial phone call script for evaluating Efficiency Nova Scotia’s Green Heat program. It includes instructions for interviewers, verification questions for respondents, and details about the program’s evaluation process.

nswer on a scale of 0 to 10, with a 0 indicating that it is “Very Unlikely,” and a 10 indicating that it is “Very Likely.” [DO NOT ACCEPT A RANGE – ASK FRS3 a-c SEQUENCE IN ORDER/DO NOT RANDOMIZE] Project No. 6138 63 Existing Residential P...

AI summary The text presents a series of questions from a program evaluation, asking participants to rate the likelihood of alternative actions if the program rebate or financing had not been available and to rate the influence of factors in their decision to install equipment.

ova Scotia prompted you to ask a contractor or a distributor about energy efficient heating systems for your home. 1. Agree 2. Disagree 98. (Don’t know) Project No. 6138 65 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evalu...

AI summary The text discusses customer experiences with Efficiency Nova Scotia's residential energy efficiency programs, focusing on satisfaction with program requirements, eligibility confirmation, rebate amounts, and overall program effectiveness. It includes survey questions to gauge customer satisfaction with the Green Heat program and the installation of energy-efficient heating systems.

l when evaluating different heating systems for your home. 1. Agree 2. Disagree 98. (Don’t know) 99. (Refused) Satisfaction and concerns [S Series] Now, I have some questions about your satisfaction with the mini-split heat pump you instal...

AI summary The text presents a survey related to customer satisfaction with the mini-split heat pump and the Green Heat program, including questions about eligibility, rebate amounts, and program clarity. It also references an evaluation report for an existing residential program in Nova Scotia.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report M4. So you considered installing a mini-split heat pump prior to 2017. What kept you from installing a mini-split heat pump in previous years? Barriers to Installing a Heat Pump...

AI summary The text discusses barriers to installing mini-split heat pumps prior to 2017, with cost being the primary barrier. It also asks about the most helpful aspect of the Green Heat program, indicating a focus on customer feedback and program evaluation.

ach group of questions were converted into a value indicating the level of program component attribution and this value was used to calculate the free-ridership level associated with each participant. For those participants who installed a...

AI summary The text discusses an algorithm used to assess free-ridership in a residential energy efficiency program, specifically for participants who installed a high-efficiency heat pump (MSHP). The algorithm weights the decision to install a heat pump at 75% and the decision to purchase a high-efficiency model at 25%, with the latter considered only if the former's free-ridership level exceeded 50%.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report Algorithm – Stoves and Solar Project No. 6138 129 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report PA4 Score: FR3c IF ABS (FR2 – MIN (FR3...

AI summary The document presents an algorithm used in the evaluation of the 2017 Demand Side Management (DSM) program by Efficiency Nova Scotia, focusing on free-ridership calculations and consistency tests involving various performance metrics.

Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report Algorithm – Air Source Heat Pump Project No. 6138 131 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report PA4 S...

AI summary The document discusses an algorithm related to the evaluation of the Existing Residential Program under Efficiency Nova Scotia's 2017 DSM Evaluation. It includes scoring metrics and inconsistency tests (PA1, PA2, PA4) used to assess free-ridership in the program.

00% had you already made the decision to install a high-efficiency heat pump IF 2. No, DK OR REF: FRM4a = 0% model? FRM5. [ASK IF FRM4=1] If you had not received the rebate from Efficiency FRM5 = Answer x 10% Nova Scotia, would you have pa...

AI summary The text outlines a survey or form (FRM) related to customer decisions regarding the installation of high-efficiency heat pumps and the influence of rebates from Efficiency Nova Scotia. It includes scoring mechanisms and consistency checks to evaluate customer responses and potential free-ridership.

ised Free-Ridership IF CI=2, Revised FR = FR .75 IF CI=3, Revised FR = FR .50 Project No. 6138 135 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report EPI: 2017 RECOMMENDATIONS This appendix summarizes...

AI summary The 2017 evaluation of Efficiency Nova Scotia's residential program highlights a decrease in LED lamp installations per household, suggesting fewer incandescent lamps are present. The Evaluator recommends tracking lamp types to assess future savings potential if other lamp types are eligible for replacement.

t of the program. Do you still have this device installed in your household? 1. Yes 2. No 98. (Don’t Know) [GO TO P SERIES] 99. (Refused) [GO TO P SERIES] E2. [ASK IF E1=2 (NO)] Why did you choose to remove this device from your household?...

AI summary The text presents survey questions related to customer satisfaction with a smart power strip installed under the Efficiency Nova Scotia program. It asks whether the device is still in use, reasons for removal, and levels of satisfaction with the device's performance.

s who installed additional products or retired appliances after participating in EPI and did not receive financing or rebates from other ENS services Don’t know/Refused excluded from calculations Project No. 6138 168 Existing Residential P...

AI summary The text discusses a residential program evaluation by Efficiency Nova Scotia, focusing on participant demographics, including types of residences and bedroom counts, as part of the 2017 DSM Evaluation.

ale 41% 43% 43% Female 59% 57% 57% Project No. 6138 172 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report EPI: ON-SITE VISIT SAMPLING METHODOLOGY AND PROTOCOL Sampling Methodology For the 2017 EPI im...

AI summary The 2017 EPI impact evaluation involved 110 on-site visits with participants, using four sampling plans between March and November 2017. The sampling aimed to include a representative mix of participants from all delivery agents and income levels, with specific criteria for installation types and numbers.

IF DK OR REF: EMPTY by at least one year? (Scale 0 to 10) PA4 Score: FR3b Project No. 6138 226 Existing Residential Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report FR3c. If the service had not been offered, what is the...

AI summary The text presents evaluation metrics and scoring criteria for a residential energy efficiency program, specifically focusing on the likelihood of LED bulb installations through the Efficient Product Installation (EPI) program. It outlines scoring scales and conditions for inconsistencies in responses.

est PA2 70%: PA2 = EMPTY MEAN VALUE OF: Free-Ridership (PA1 ; PA2 ; PA3 ; PA4; PA5) CI2/CI4. [Your previous participation in an ENS program or service and what you learned by participating in this program]/[The energy efficiency CI = MAX (...

AI summary This document discusses the evaluation of an existing residential energy efficiency program by Efficiency Nova Scotia (ENS) as part of the 2017 DSM Evaluation. It includes a free-ridership calculation based on participant responses and revised weighting factors based on participant influence scores.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text defines key terms related to energy efficiency program evaluation, including gross and net savings, free-ridership, induced consumption, and interactive effects. It outlines the evaluation plan and first-year savings as central components of the assessment process.

program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to programs through which rebates are offere...

AI summary The text defines several key metrics related to energy efficiency programs, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These metrics are used to evaluate the effectiveness and impact of energy efficiency initiatives.

Errors arising during the course of all survey activities other than sampling. Non-sampling error Non-sampling errors tend to lead to bias in measurements and are very difficult to quantify. Peak coincidence The percentage of measure deman...

AI summary The text discusses various statistical and measurement concepts in energy efficiency programs, including non-sampling errors, peak coincidence, precision, sample size, secondary market impacts, tracked savings, and unitary savings. These concepts are relevant to the evaluation and measurement of energy savings in demand-side management initiatives.

e annual energy use in gigajoules. NHC’s incentive structure has since been adapted to accommodate this change by using this new EnerGuide rating system, but the rebate amounts have remained the same. In October 2016, Efficiency Nova Scoti...

AI summary The 2017 evaluation of the New Home Construction (NHC) program and the Passive House pilot found that combined net energy savings were 3.371 GWh and net peak demand savings were 0.954 MW. Participation in NHC was lower in 2017, but per-participant energy savings were the highest since 2011.

Energy Savings (GWh) Net Energy Savings per Participant (MWh) Peak Demand Savings (MW) Market Evaluation Highlights The participants and builders interviewed expressed moderately high satisfaction ratings for NHC, respectively averaging 6....

AI summary The evaluation of the New Home Construction (NHC) program highlights moderate satisfaction among participants and builders, with dissatisfaction primarily attributed to low rebate amounts and complex processes. Changes in participation measurement methods starting in 2013 also led to discrepancies in reported participant numbers across evaluation reports.

and Evaluated Savings at the Generator for New Residential Initial Gross Savings Adjusted Gross Savings NTGR Net Savings Energy Savings Tracked Savings from ENS 3.845 GWh 3.217 GWh 0.93 2.992 GWh Evaluation Results 3.838 GWh 3.212 GWh 1.05...

AI summary The evaluation of energy savings from the New Residential program shows slight decreases in gross savings but higher net savings due to changes in the NTGR value. The 2017 evaluation found that the program met peak demand savings targets but fell short of net energy savings goals, with participation at its lowest since 2011. Recommendations for improvement are provided.

2017 NHC-R1. Conduct a process evaluation for NHC in 2018. During the 2017 evaluation, participants and builders made certain criticisms about NHC, especially on incentive levels and the participation process. Also, the NHC participation l...

AI summary The document recommends conducting a process evaluation for the New Home Construction (NHC) program in 2018 to assess its effectiveness and optimize its delivery model. It also suggests a billing analysis to determine the overestimation ratio for energy savings using the new version of HOT2000, following increased data availability.

entation status of each recommendation presented in the executive summary section of the 2016 evaluation report. Table 5: Implementation Status of the Recommendations in the 2016 Executive Summary 2016 Recommendations Status 2016 NHC-R1. C...

AI summary The 2016 evaluation report's recommendations related to New Home Construction (NHC) have been implemented. These included continuing project reviews and ensuring complete documentation in participant files. ENS has taken steps to improve consistency through training and process reviews, especially after transitioning to a new rating system in 2017.

3.212 0.909 2011 2012 2013 2014 2015 2016 2017 2011 2012 2013 2014 2015 2016 2017 New Home Construction 6 New Residential Program Efficiency Nova Scotia 2017 DSM Evaluation Report 2 NHC EVALUATION METHODOLOGY This section presents the meth...

AI summary This section outlines the methodology used in the 2017 New Home Construction (NHC) evaluation by Efficiency Nova Scotia. It describes the data collection activities, including a review of program documentation, a meeting with the program manager, and the preparation of data collection instruments.

instruments. Once the program manager meeting and the program component documentation review were completed, specific evaluation activities were undertaken, as described in the following subsections. For evaluation activities that yield qu...

AI summary The document outlines the evaluation process for the Efficiency Nova Scotia program, focusing on the quantitative evaluation activities with a 10% margin of error at a 90% confidence level. It distinguishes between precision and accuracy and notes that the evaluation accounts only for random sampling errors, not non-sampling errors.

Efficiency Nova Scotia 2017 DSM Evaluation Report As part of this evaluation, the Evaluator calculated the margins of error for survey samples and free-ridership levels. The margins of error are presented alongside the results in the relev...

AI summary This section of the 2017 DSM Evaluation Report discusses the survey methodology used to evaluate Efficiency Nova Scotia's programs. A telephone survey was conducted with 13 NHC homeowner participants to gather feedback on program participation, satisfaction, free-ridership, barriers, and recommendations. Due to the small population of available homeowners, a census approach was used, resulting in a 20.9% margin of error at a 90% confidence level.

leted interviews out of the 27 available contacts. Drawing a random sample of 13 from a population of 57 would yield a margin of error of 20.9 percent at a 90 percent confidence level. Builder Survey In November 2017, CRA conducted a telep...

AI summary The document discusses survey methodologies and results from a builder survey and Passive House pilot participant interviews conducted in 2017. The builder survey aimed to assess free-ridership and spillover effects, while the Passive House interviews focused on evaluating free-ridership among participants. Surveys were conducted by CRA with a margin of error calculated for both studies.

meowners, and one was a builder. The main purpose of the interviews was to assess free-ridership. The interview guide and results are presented in Appendices VI and VII. Effective Useful Life Review As part of the 2017 evaluation, to exami...

AI summary The document discusses interviews conducted to assess free-ridership in energy efficiency programs and an evaluation of effective useful life (EUL) values used by Efficiency Nova Scotia (ENS) in calculating lifetime energy savings. It also presents a survey of homeowners under the New Home Construction (NHC) program.

ipation process simpler (2 mentions), and offering more upgrades eligible for rebates (2 mentions). Finally, it is important to note that half the participants were unable to make any recommendations. As demonstrated in Figure 5 below, sat...

AI summary The evaluation report highlights participant satisfaction with aspects of the New Home Construction (NHC) program, particularly the time allowed to complete the project and the energy efficiency reports. However, satisfaction with the first review of house plans and the rebates provided was lower, indicating areas for improvement.

(n=11) Final energy assessment (n=11) 7.7 First review of house plans overall (n=12) 6.8 Rebates provided by the program component overall 5.2 (n=13) 3.2 Builder Perspectives A survey of 40 participant builders and 19 non-participant build...

AI summary The survey of builders participating in the New Home Construction (NHC) program found that most participants make the decision to join alone, with satisfaction at 7.2 out of 10. Key issues include low rebate amounts, lack of information, and delays in receiving rebates. Non-participants cited low rebates and program complexity as barriers to participation.

s Factor 1.100 Total Gross Peak Demand Savings – at the Generator (MW) 0.014 4.2 Net-to-gross Ratio The net-to-gross ratio (NTGR) serves to determine net savings, i.e. the energy savings that can be reliably attributed to a program compone...

AI summary The document discusses the net-to-gross ratio (NTGR) and its role in determining net energy savings from program components, particularly focusing on free-ridership. It explains that free-ridership occurs when participants would have implemented energy efficiency measures without the program and outlines the methodology used to assess this, including telephone surveys and algorithms detailed in Appendix VIII.

cantly vary from one evaluation to another. Table 11 lists the measured free-ridership levels obtained for each respondent type. Table 11: Measured Free-ridership Levels Average Sample Population Margin of Freeridership Level Size Size Err...

AI summary The document evaluates free-ridership and spillover effects in energy efficiency programs. It reports free-ridership levels for different respondent groups and calculates the NTGR based on these metrics, indicating a 106% NTGR for NHC.

Tracked Savings 0.021 MW 0.014 MW 0.93 0.013 MW Evaluation Results 0.021 MW 0.014 MW 0.39 0.006 MW The savings tracked by ENS were compared with the revised results established by the Evaluator for 2017. For NHC, the evaluated gross energy...

AI summary The 2017 DSM Evaluation Report compares tracked savings with evaluated results for the New Home Construction (NHC) and Passive House pilot programs. Differences in savings are attributed to changes in the NTGR value and free-ridership levels, impacting net energy and peak demand savings.

Efficiency Nova Scotia 2017 DSM Evaluation Report CONCLUSION The New Residential program, including NHC and the Passive House pilot, achieved 3.371 GWh in net energy savings and 0.954 MW in net peak demand savings at the generator, which r...

AI summary The 2017 DSM Evaluation Report highlights the performance of Efficiency Nova Scotia's New Residential program, including NHC and the Passive House pilot. The program achieved significant energy and peak demand savings, though participation and savings decreased compared to 2016. Free-ridership and spillover levels were also analyzed, with the Passive House pilot showing a high free-ridership rate.

stablished at 61 percent, which is fairly high. All but one interviewed participant mentioned that they would have built or bought a passive house even if the Passive House pilot had not been offered. Participants and builders reported mod...

AI summary The New Residential Program evaluation report highlights high satisfaction with NHC among participants and builders, though dissatisfaction arose due to low rebate amounts and lack of information. The report includes appendices with survey results, interview guides, and methodologies for free-ridership and spillover calculations.

se Pilot Participant Interview Guide Appendix VII - Passive House Pilot Participant Interview Results Appendix VIII - Algorithm for Free-ridership Calculation Appendix IX - Spillover Methodology New Residential Program 29 160 Saint-Paul St...

AI summary This document provides appendices related to the evaluation of the New Residential Program under Efficiency Nova Scotia's 2017 DSM (Demand Side Management) Evaluation. It includes interview guides, free-ridership calculation algorithms, and spillover methodology.

Sections Recommendations 1. Conduct a process evaluation for NHC in 2018. During the 2017 evaluation, participants and builders made certain criticisms about NHC, especially on incentive levels and the participation process. Also, the NHC...

AI summary The text recommends conducting a process evaluation for the New Home Construction (NHC) program in 2018 due to declining participation and criticisms about incentive levels and the participation process. It also suggests a billing analysis to review the overestimation ratio using updated data and tools like HOT2000.

consumption value available. Now that energy consumption is available for more participants and a new version of HOT2000 is being used to calculate energy savings, the Evaluator recommends conducting a new billing analysis in the next eval...

AI summary The text discusses the need for a new billing analysis to evaluate energy savings under the New Home Construction program using the updated HOT2000 tool. It also outlines a participant survey questionnaire for evaluating Efficiency Nova Scotia’s program.

ative about your experience with the program. This survey is different as it focuses on other aspects of your participation in the program.) (IF NEEDED: The survey should take about fifteen minutes.) Verification and Recall (V Series) V1....

AI summary This text describes a survey process related to the New Home Construction program, focusing on verifying participation and collecting feedback from individuals or businesses involved in building or selling homes. It includes prompts and branching logic based on responses.

ou during the New Home Construction program review of home plans that you chose not to install in your new home? 1. Yes 2. No 98. (Don’t recall) Project No. 6138 9 New Residential Program Efficiency Nova Scotia 2017 DSM Evaluation - Append...

AI summary The text includes survey questions about participation in the New Home Construction program and barriers to implementing energy efficiency upgrades, as well as a section for recommendations for program improvements.

19 13 Mean 7.1 6.6 Don’t know/Refused excluded from calculations Project No. 6138 14 New Residential Program Efficiency Nova Scotia 2017 DSM Evaluation - Appendix Report S2. What was the most important reason you were not more satisfied wi...

AI summary The text provides a statistical summary and a question from an evaluation of the 2017 DSM (Demand Side Management) program, focusing on customer satisfaction and reasons for dissatisfaction with the program.

ERGY STAR/R2000 label/certification) 96. (Other) [SPECIFY _]) 98. (Don’t know) 99. (Refused) Satisfaction with Program (S Series) [ASK PARTICIPANT BUILDERS ONLY] S1. Using a scale from 1 to 10 where 1 is “not at all satisfied” and 10 is “c...

AI summary The text outlines survey questions related to participant satisfaction with the New Home Construction program, including rating satisfaction, reasons for dissatisfaction, and recommendations for improvement. It also references a project number and an evaluation report related to a residential energy efficiency program in Nova Scotia.

- 7% - Lower the EnerGuide rating needed for rebate/incentive - 4% - Include other forms of energy 14% - - Lengthen the time frame to register 5% - - Others 10% 15% 15% No recommendations 19% 37% 35% Multiple Responses Project No. 6138 44...

AI summary The text presents a survey or evaluation related to a residential energy efficiency program, including percentages of responses to various recommendations and a question about the number of full-time equivalent workers employed by an organization within Nova Scotia.

was used for the Passive House pilot, with the main difference being that the information was collected through interviews instead of surveys due to the low number of participants. NHC - Builders Project No. 6138 55 New Residential Program...

AI summary The text discusses a residential energy efficiency program evaluated by Efficiency Nova Scotia in 2017, focusing on participant responses to questions about rebate impact and likelihood of building energy-efficient homes without the program.

MEAN VALUE OF: Free-Ridership (PA1; PA2; PA3) Project No. 6138 57 New Residential Program Efficiency Nova Scotia 2017 DSM Evaluation - Appendix Report SPILLOVER METHODOLOGY The marketing of New Home Construction is primarily conducted thro...

AI summary The text discusses the spillover methodology used in evaluating the New Residential Program by Efficiency Nova Scotia. It explains how program promotion can influence builders to construct energy-efficient homes even without participating in the program, and how an average attribution level was established to quantify this effect.

ipating in the program. An average attribution level was then established among the builders to establish the percentage of non-participating eligible houses built due to the influence of the program. To calculate the number of non-partici...

AI summary The evaluation calculates the number of non-participating eligible houses influenced by the program using data from the 2017 tracking sheet and Canada Mortgage and Housing Corporation data. It involves determining the total number of houses built, subtracting participating houses, and applying an average attribution level.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text outlines key terms and concepts related to energy efficiency program evaluation, including gross and net savings, free-ridership, induced consumption, and interactive effects. It emphasizes the importance of evaluation plans and the first-year savings metric in assessing program effectiveness.

d by a program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to programs through which rebates are...

AI summary This text defines several key terms related to energy efficiency programs and their measurement, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These terms are used to evaluate the effectiveness and impact of energy efficiency initiatives.

d LED Products from 2015 to 2017 .................................. 67 Figure 15: Market Evolution for Commercial LED Lighting ................................................................... 71 Efficient Product Rebates Program x Effic...

AI summary This report evaluates the 2017 demand-side management results for the Efficient Product Rebates program, including the Business Energy Rebates component. It details participation levels and savings from both mail-in and instant rebate services.

he rebate. BER Mail-in succeeded in maintaining high satisfaction scores with all these service aspects for a second year in a row, indicating sustained improvement in Mail-in satisfaction since 2015. The results of the site visits and pro...

AI summary The BER Mail-in program maintained high satisfaction scores, with only minor adjustments needed for tracked energy savings. The NTGR was set at 0.86 based on free-ridership and spillover effects. For BER Instant Rebates, there was a 7% increase in rebated products sold in 2017, and overall satisfaction remained consistent with previous years.

d an overall positive opinion (ratings of 8 or higher), one distributor offered a moderate opinion (rating of 7) and three distributors were critical of BER Instant Rebates (ratings of 5 or lower). Efficient Product Rebates Program xiii Ef...

AI summary The 2017 DSM Evaluation Report highlights dissatisfaction with the Efficient Product Rebates Program, particularly due to rebates being directed to contractors rather than end users. However, satisfaction with Efficiency Nova Scotia's support and communication is high, with an average satisfaction level of 8.3.

gories. Free-ridership levels for LED products vary between 32 and 38 percent. A NTGR of 1.00 was applied to the other product categories, resulting in an overall NTGR of 0.73 for BER Instant Rebates. Table 3: Net-to-Gross Ratio for BER In...

AI summary The document discusses the application of net-to-gross ratios (NTGR) to calculate net savings for the Efficient Product Rebates Program, specifically for LED products and other categories. Free-ridership levels and NTGR values are provided, with an overall NTGR of 0.73 for BER Instant Rebates. The evaluation process for 2017 savings is outlined.

oth services. Overall, the evaluated net energy savings are 8 percent lower than the tracked values as a result of both Mail-in and Instant Rebates being adjusted downward (3% and 11% respectively). For Mail-in, due to efforts made by ENS...

AI summary The 2017 DSM Evaluation Report highlights that net energy savings from Mail-in and Instant Rebates are 8% lower than tracked values, due to adjustments made by ENS based on the 2016 evaluation. Net peak demand savings are 34% lower, with significant reductions in both Mail-in and Instant Rebates due to factors like free-ridership and changes in the net-to-gross ratio.

Efficiency Nova Scotia 2017 DSM Evaluation Report BER Market Evolution Findings As part of the 2017 evaluation, the Evaluator conducted a market analysis of the LED product market in the BNI sector in Nova Scotia. The Evaluator analyzed se...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia highlights the significant growth in LED technology adoption in the BNI sector, with market share increasing from 0% in 2010 to 54% in 2016. Prices have declined, and LEDs are widely available. Upcoming regulations will further increase efficiency requirements, with LEDs being the most efficient option available.

s available. With this information, the Evaluator came to the conclusion that LEDs continue to be in a growth phase, but are approaching maturity as the BNI sector increasingly adopts these products. BER Recommendations Overall, the Evalua...

AI summary The Evaluator concludes that the Business Energy Rebates (BER) program generated significant savings, with stable participation in BER Mail-in and increased sales and savings in BER Instant Rebates. Recommendations are provided to optimize the program.

2017-BER-R1. Continue to monitor the market evolution of LED lamps and adapt the BER offer when needed. The 2017 evaluation served to confirm solid market evolution. The high number of LEDs sold during the 2017 year, their declining retail...

AI summary The 2017 evaluation of the Business Energy Rebates (BER) program highlights the decreasing market barriers for LED lamps and the need to monitor market evolution. It also emphasizes the need to diversify the BER Mail-In portfolio, as lighting projects dominate savings, and the risk of declining savings due to increased adoption of LED technology.

uator recommends that ENS identify BNI sectors that are currently underrepresented in the Mail-in service and that could potentially generate significant savings. 2017-BER-R3. Continue and improve the tracking of BER Instant Rebates partic...

AI summary The evaluator recommends that Efficiency Nova Scotia (ENS) identify underrepresented BNI sectors in the Mail-in service and improve tracking of BER Instant Rebates participants to refine impact analysis and better estimate annual hours of use and free-ridership.

fficiency codes and standards introduced in Nova Scotia. To carry out this assignment, Econoler collaborated with Research Into Action, Equilibrium Engineering and Corporate Research Associates (CRA). Econoler served as team leader in the...

AI summary The text discusses the evaluation of energy efficiency programs in Nova Scotia, conducted by Econoler and its partners. The evaluation included impact assessments, process evaluations, and data collection activities. The 2017 evaluation involved reduced evaluation activities for certain programs, such as Instant Rebates.

role in not only promoting Instant Rebate measures to end users, but also providing customer service to end users in the form of product expertise and assistance in completing mail-in applications. Business Energy Rebates 3 Efficient Produ...

AI summary The Business Energy Rebates (BER) program was promoted through various channels in 2016, with goals of achieving 37.2 GWh in net electricity savings and 5.9 MW in net peak demand savings by 2017. A follow-up evaluation of the BER component from 2016 is discussed, along with the implementation status of recommendations made by the Evaluator.

Rx Screening Tool,3 as were the values used for heat pump full load hours. Additionally, new procedures for baseline wattages in new construction projects and in Instant Rebates have been established. Also, ENS has improved the BER Mail-in...

AI summary Efficiency Nova Scotia (ENS) has updated procedures for baseline wattages in new construction projects and Instant Rebates. Additionally, ENS improved the BER Mail-in internal review process and implemented a new M&V protocol to enhance accuracy in tracked savings. Market analyses on LED lighting were conducted as part of the 2017 DSM Evaluation Report.

is section presents the methodology used and the activities carried out for the BER 2017 evaluation. As illustrated in Figure 7, various data-collection activities were used throughout the evaluation. Figure 7: Methodological Model The Eva...

AI summary This section outlines the methodology used for the 2017 evaluation of the Business Energy Rebates (BER) program. It describes the data collection activities, including a review of BER documentation and a meeting with the program manager in August 2017 to verify the implementation of previous recommendations and understand recent changes. The evaluation aimed for a 10% margin of error at a 90% confidence level for quantitative results.

s, the margin of error indicates the relative level of precision6 of the measurement, while its associated confidence level indicates the probability of a measure falling within this margin of error.7 As part of this evaluation, the Evalua...

AI summary The evaluation of the BER Mail-in program includes a survey of 70 participants to assess participation reasons, free-ridership, satisfaction, and barriers. A margin of error of 8.1 percent at a 90 percent confidence level was calculated for the sample, considering only random sampling errors.

ke random sampling errors into account; they do not reflect non-sampling errors or biases such as data entry errors, inaccurate responses from respondents or the inability to contact certain people. Business Energy Rebates 10 Efficient Pro...

AI summary The 2017 DSM Evaluation Report discusses on-site visits conducted to validate energy efficiency measures and assess spillover effects, as well as a review of effective useful life values used in calculating energy savings. The report includes a participant survey and references technical reference manuals.

gy savings, the Evaluator conducted a thorough literature review by consulting technical reference manuals (TRMs), evaluation reports and relevant studies. 2.2 Instant Rebates Participant Survey In November 2017, CRA conducted a telephone...

AI summary The Evaluator conducted a literature review using technical reference manuals and evaluation reports. A survey was conducted with 60 participants of the BER Instant Rebates program to gather feedback on participation reasons, free-ridership, cross-influence, and decision criteria. The survey had a 10.6% margin of error at a 90% confidence level.

Efficiency Nova Scotia 2017 DSM Evaluation Report Interviews with Distributors In November 2017, CRA conducted eight interviews with participating distributors for BER Instant Rebates. These interviews were mainly used to establish free-ri...

AI summary Efficiency Nova Scotia conducted interviews with distributors in 2017 to assess the BER Instant Rebates program, focusing on free-ridership and satisfaction. Savings and effective useful life (EUL) values were reviewed, with the 2017 evaluation scope defined as 'reduced' due to unchanged assumptions from the 2016 evaluation.

evaluation objectives, key results of the previous evaluations and recent changes made to each program. Further explanations can be found Section 1 of the DSM Programs Evaluation Executive Summary. Business Energy Rebates 12 Efficient Prod...

AI summary The 2017 BER market evaluation highlights participant satisfaction with the Mail-in Rebates program, with 90% satisfaction ratings. Participants primarily joined to save on energy costs, and many had previously engaged with other Efficiency Nova Scotia programs. Some dissatisfaction was noted due to paperwork and lengthy processes.

Efficiency Nova Scotia 2017 DSM Evaluation Report Nearly one participant in ten (9%) had issues with measures installed through BER Mail-in, which mainly related to equipment malfunctions. In all cases, the problems were resolved and parti...

AI summary The 2017 DSM Evaluation Report highlights that 9% of participants had issues with BER Mail-in measures, primarily due to equipment malfunctions, but all were resolved. Participants suggested improvements like more user-friendly forms, higher rebates, better promotion, and more information. Most participants in Instant Rebates were motivated by energy cost savings and had prior experience with ENS programs. Distributors were generally satisfied with the service and its market influence.

he factors contributing to the increase of LED technology adoption. 8 The regulations and legislation factor is not included here because it does not reflect the influence of ENS on the market. Business Energy Rebates 17 Efficient Product...

AI summary The 2017 BER Mail-in impact evaluation assesses the gross and net energy savings from the Business Energy Rebates program. ENS tracks gross savings using the CIRx Screening Tool and data from on-site visits, with 288 projects across 11 measure categories. Each measure, such as LED fixtures or HVAC sensors, is tracked individually.

uantity of cows listed because the participant declared the maximum capacity of the barn and not the current average number of cows milked per day. This led to a 30 percent decrease in energy savings. The second adjustment was to the PCF u...

AI summary Adjustments were made to energy savings calculations for a barn project and commercial kitchen measures due to inaccuracies in data inputs. These adjustments significantly impacted overall category savings, with the barn project accounting for 98% of the category’s total savings. The Evaluator recommended providing specific tools for future accuracy.

Efficiency Nova Scotia 2017 DSM Evaluation Report All measures were used to calculate the energy savings adjustment ratio, and one measure was removed from the peak demand savings ratio due to the abovementioned typographical error. The ad...

AI summary The 2017 DSM Evaluation Report details the calculation of energy savings adjustment ratios, with specific measures removed or adjusted due to typographical errors. Adjustments were made for energy savings and peak demand savings, with no adjustments needed for several commercial measures.

1.066 1.066 1.066 - Gross Peak Demand Savings – at the Generator (MW) 0.023 0.010 0.004 0.001 0.000 2.863 Business Energy Rebates 29 Efficient Product Rebates Program Efficiency Nova Scotia 2017 DSM Evaluation Report 4.2 Net-to-gross Ratio...

AI summary This section discusses the net-to-gross ratio (NTGR) used to determine net savings from energy efficiency programs, specifically addressing free-ridership in the BER Mail-in program. Free-ridership occurs when participants would have implemented energy efficiency measures without the program, assessed through a self-report telephone survey.

NS Tracked Savings 3.505 MW 3.505 MW 0.84 2.944 MW Evaluation Results 3.505 MW 2.863 MW 0.86 2.462 MW The gross evaluated energy savings are 5 percent lower than the tracked values. While the Evaluator applied a very small adjustment to th...

AI summary The evaluation of the BER Instant Rebates program found that gross evaluated energy savings were 5% lower than tracked values, primarily due to a 7.3% reduction in lighting savings. Net energy savings were 3% lower, while net peak demand savings were 16% lower due to an 18% adjustment to an HVAC project.

ible NTGR estimate.27 The evaluation targets the following four specific products: › LED linear fixtures › LED downlight fixtures › LED linear lamps › LED general-use and decorative lamps These product categories represent 75 percent of al...

AI summary The evaluation focuses on four LED product categories that account for 75% of energy savings from BER Instant Rebates. These products were selected due to their high sales volume and potential for free-ridership. Interviews with distributors and end-users were conducted to assess the influence of the service on market behavior and purchasing decisions.

Efficiency Nova Scotia 2017 DSM Evaluation Report Other lamp and fixture types are also affected by Canadian energy efficiency regulations. Amendment 13 has introduced new requirements for the efficiency of fluorescent linear lamp ballasts...

AI summary The 2017 DSM Evaluation Report discusses the impact of Canadian energy efficiency regulations on lighting products, noting that while amendments are evolving, they do not significantly improve efficiency levels. The report also evaluates free-ridership levels in the LED BNI market in Nova Scotia.

recommends that ENS identify BNI sectors that are currently underrepresented in the Mail-in service and that could potentially generate significant savings. 3. Continue and improve the tracking of BER Instant Rebates participants. The 2017...

AI summary The document recommends that ENS identify underrepresented sectors in the Mail-in service and improve tracking of BER Instant Rebates participants to better assess savings and refine impact analysis.

reviewing applications for return and exhaust measures and identifying if projects achieve savings from VFDs through controls or schedule changes. Impact Evaluation 6. Six commercial kitchen projects were reviewed for a total of 16 measure...

AI summary The text discusses the review of commercial kitchen projects and the need for adjustments in energy savings calculations, particularly concerning VFDs and operational hours. It also mentions a mail-in participant survey for BER, highlighting the importance of stakeholder engagement in energy efficiency programs.

were a major factor in your company’s decision to implement energy-efficient [MEASURE CATEGORY] measures. 1 Agree 2 Disagree 98 (Don’t know) 99 (Refused) CI4b. [IF CI3=1] The energy-efficiency promotional materials distributed by Efficienc...

AI summary The text includes survey questions related to the impact of Efficiency Nova Scotia's energy-efficiency promotional materials on company representatives' decisions to implement energy-efficient measures and evaluate their cost-effectiveness. It also references the Business Energy Rebates program and a satisfaction survey.

Efficiency Nova Scotia 2017 DSM Evaluation Report BER : INSTANT REBATES PARTICIPANT SURVEY QUESTIONNAIRE Could I speak with the person in your organization who would have been involved with the Efficiency Nova Scotia Business Energy Rebate...

AI summary This document is a survey questionnaire from Efficiency Nova Scotia's 2017 DSM Evaluation Report, targeting participants in the Business Energy Rebates (BER) Program. It aims to gather feedback on the purchase of energy-efficient products and the rebate process.

Efficiency Nova Scotia 2017 DSM Evaluation Report CI2b. Because of the company’s previous participation in an Efficiency Nova Scotia program and what was learned by participating in the program, a company representative asked a contractor...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia shows that 76% of respondents who previously participated in an ENS program agreed that their participation influenced them to seek efficient lighting options. Additionally, 84% of respondents considered the cost-effectiveness of products when evaluating options for their facilities.

R: INSTANT REBATES INTERVIEW GUIDE WITH DISTRIBUTORS Interview Date and Time Company name Name Area code and phone number Email Interview Length Introduction Thank you for taking the time to speak with me. Today, I am interested in your op...

AI summary This document is an interview guide for evaluating Efficiency Nova Scotia’s Business Energy Rebates program. It outlines the process for gathering feedback from distributors regarding their involvement in the program, focusing on the stocking and supplying of specific energy-efficient products.

Efficiency Nova Scotia 2017 DSM Evaluation Report BER: ALGORITHM FOR FREE-RIDERSHIP CALCULATION The figures below present the algorithms for calculating the free-ridership level of BER Mail-in and Instant rebates measures. The participant...

AI summary The document discusses the calculation of free-ridership levels for BER Mail-in and Instant rebates measures through algorithms and participant surveys. The surveys assess various aspects of the program, including planning, quantity, efficiency, timing, cost, and influence, which are used to determine program attribution and free-ridership levels.

Efficiency Nova Scotia 2017 DSM Evaluation Report BER: ALGORITHM FOR INTERNAL SPILLOVER CALCULATION Internal spillover for BER Mail-in was measured using a participant survey conducted during the on-site visits. Participants were asked whe...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia discusses the methodology used to calculate internal spillover for the BER Mail-in program. It involved surveying participants to determine if they implemented additional energy efficiency measures outside of the program and quantifying the program's influence on these actions.

active Effects Factor (demand) 0.7 Industrial Bldg. with Electrical Heating Annual Energy Consumption (kWh) 12,303 Calculation Annual Peak Demand (kW) 0 Calculation Revised Savings Revised Energy Savings (kWh) 7,269 Calculation Revised Dem...

AI summary This document outlines the evaluation of the Custom Incentives Program under Efficiency Nova Scotia's 2017 Demand Side Management (DSM) initiative. It includes calculations for energy and demand savings, an adjustment ratio algorithm, and references to a final report dated March 15, 2018.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text outlines key terms and concepts related to energy efficiency program evaluations, including gross and net savings, free-ridership, induced consumption, and interactive effects. It describes the evaluation process and the components of an evaluation plan, emphasizing first-year savings and the net-to-gross ratio.

efficiency measure introduced by a program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to progra...

AI summary The text defines several key terms related to energy efficiency programs, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These terms help quantify the effectiveness and impact of energy efficiency measures and programs.

, that negatively or positively affect the energy savings attributable to a program. Net-to-gross ratio The ratio of net energy savings to gross energy savings. Custom Incentives Program iv Custom Incentives Program Efficiency Nova Scotia...

AI summary The text introduces the Custom Incentives Program under Efficiency Nova Scotia and references the 2017 DSM Evaluation Report, focusing on metrics such as net-to-gross ratio, which measures the effectiveness of energy savings from programs.

............................. 60 13.2.5 Revised Gross Savings .................................................................................................. 63 13.3 Net-to-gross Ratio ......................................................

AI summary The document outlines the structure of the 2017 DSM Evaluation Report for the Custom Incentives Program by Efficiency Nova Scotia, including sections on gross savings, net-to-gross ratio, and overall savings. It also includes a list of tables summarizing participation and savings data.

Efficiency Nova Scotia 2017 DSM Evaluation Report Suggestions for Retrofit Improvement Perhaps because of the high levels of satisfaction with Retrofit, respondents made few suggestions on how to improve the service. A small number of resp...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia highlights improvements in the Retrofit program, including enhanced project documentation and a closer adjustment ratio to unity. Minor adjustments were made to energy savings calculations, and two projects required special corrections due to implementation issues. Overall, the program saw positive outcomes from implementing 2016 evaluation recommendations.

tor from thoroughly revising the tracked savings of these projects, which accounted for 19.4 percent of the tracked energy savings and 15.9 percent of the peak demand savings of the sampled projects. Custom Incentives Program xii Custom In...

AI summary The 2017 DSM Evaluation Report discusses the net-to-gross ratio (NTGR) for the Retrofit program, using historical free-ridership and spillover data. It calculates energy and peak demand savings, along with associated CO2 emissions reductions and lifetime energy savings.

Efficiency Nova Scotia 2017 DSM Evaluation Report The Evaluator conducted in-depth technical review and on-site visits for four of the six Building Optimization projects completed in 2017, which represented all projects completed at the ti...

AI summary An evaluation of the 2017 Building Optimization projects found that documentation gaps and modeling inaccuracies led to significant reductions in reported energy and peak demand savings. The net energy savings for the year were 0.836 GWh, with an estimated 4.599 GWh in net lifetime energy savings based on an average EUL of 5.5 years.

Efficiency Nova Scotia 2017 DSM Evaluation Report The evaluated net savings for Custom are 8 percent below the tracked value. This reduction was mostly due to New Construction, which had its gross energy savings reduced by 27 percent this...

AI summary The 2017 DSM Evaluation Report indicates that Custom program net savings are 8% below tracked values, primarily due to a 27% reduction in New Construction energy savings. Retrofit contributed 70% of Custom’s gross savings with minimal adjustments. The report recommends improvements to EMIS and highlights the need for greater implementation of New Construction and Building Optimization services.

2017-Custom-R2. Continue conducting an in-depth review of energy models as part of the evaluation and improve ENS staff’s capacity to identify discrepancies in energy models. The more in-depth review of energy models conducted during the 2...

AI summary The 2017 evaluation found discrepancies in energy models across five projects, leading to significant downward adjustments in savings. Issues were particularly notable in heat pump system modeling and baseline calculations. The Evaluator recommends a systematic review of energy models and improved staff training. Additionally, Building Optimization projects varied widely in scope, and the Evaluator suggests establishing clear objectives to better align support with participants' needs.

tainability plan, help to involve participating facilities’ maintenance staff in the process, and that ENS could provide more support and manpower during the installation process. EMIS Energy Savings In 2016, the Evaluator made a distincti...

AI summary The document discusses the evaluation of energy savings from four projects in 2017, emphasizing the distinction between cumulative and incremental savings. The Evaluator recommended tracking incremental savings, which reflect new or improved energy efficiency measures, rather than cumulative savings. This approach provides more accurate impact results for the program component.

n year or from recent improvements to energy efficiency measures that were initiated in previous evaluation years. These savings are calculated by using the previous year’s situation as the baseline. Custom Incentives Program xviii Custom...

AI summary The Evaluator found that the documentation for the Custom Incentives Program was sufficient but noted the absence of monthly logs to track changes affecting baseline or incentivized projects. Discrepancies between final tracking and annual reports were also identified, with a recommendation to align future tracking with annual reports.

the final tracking and the savings reported in the final annual reports. The Evaluator recommends that from now on, the final tracking sheet be updated with the values presented in the annual reports. As part of the cumulative savings revi...

AI summary The Evaluator recommends updating tracking sheets with annual report data and adjusting baseline calculations for energy savings projects. Adjustments were made for compressed-air projects, and production data should be tracked as an independent variable. Incremental savings calculations were reviewed, and peak demand savings were identified for additional projects.

in peak demand savings. For the two projects with incremental savings calculated by the service provider, the Evaluator found that the incremental baselines used by the service provider were adequate. Free-ridership and spillover effects w...

AI summary The document evaluates the energy savings from the Custom Incentives Program under EMIS. It confirms that the incremental baselines used by the service provider were adequate and assumes no free-ridership or spillover effects due to limited market penetration. The net savings were equal to the gross savings, as confirmed by participant interviews.

Tracked Savings 0.137 MW 0.137 MW 1.00 0.137 MW Evaluation Results 0.137 MW 0.255 MW 1.00 0.255 MW In 2017, the Evaluator did not adjust the incremental savings of the two projects for which a proper incremental baseline approach was used...

AI summary The 2017 DSM Program Evaluation Report highlights a 20% reduction in energy savings due to changes in evaluation methods and the addition of peak demand savings to two projects. The Evaluator found that EMIS was effectively delivered, with high participant satisfaction and additional savings in the second or third year of participation.

Efficiency Nova Scotia 2017 DSM Evaluation Report 2017 EMIS-R1. Ensure that the tracking sheet is complete and accurate. While reviewing the project documentation, the Evaluator found several discrepancies between the savings figures repor...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia found discrepancies between savings figures in the tracking sheet and annual reports. It also noted that peak demand savings were not tracked in the tracking sheet and recommended updating the tracking sheet with final values from annual reports.

tions The process evaluation specifically targets the Retrofit service of the Custom program component. Due to the start date of the latest Strategic Energy Management (SEM) cohort, SEM did not have any participants who achieved savings in...

AI summary The evaluation report focuses on the Custom Incentives program, specifically the Retrofit service, and includes market and impact evaluations for the Custom and EMIS components. A process evaluation was added for Retrofit in 2017 due to its significant savings and lack of recent review.

ion Application, a Master List of Findings and an Implementation Application. Measurement and verification (M&V) must be performed and accounted for in an M&V report submitted by the service provider. For all three services, Business Devel...

AI summary The text outlines requirements for measurement and verification (M&V) reports in energy efficiency programs, explains how savings are claimed for different project types, and mentions follow-up actions on 2016 evaluation report recommendations for the Custom program component.

data collection activities carried out for each Custom service, namely Retrofit, New Construction and Building Optimization. Figure 7: Methodological Model Program Documentation Review Interview with Program Manager Retrofit New Constructi...

AI summary The evaluation process involved reviewing program documentation, meeting with the program manager, and conducting interviews and site visits for different service components. The goal was to assess implementation status and gather data on changes made to the program component.

e prepared. Once the program manager meeting and the program component documentation review were completed, specific evaluation activities were undertaken, as described in the following sub-sections. For evaluation activities that yield qu...

AI summary The text discusses the evaluation activities conducted for the Custom Incentives Program under Efficiency Nova Scotia's 2017 DSM Evaluation Report. It outlines the statistical approach used, aiming for a 10% margin of error at a 90% confidence level, while noting that this only accounts for random sampling errors and not biases or data entry issues.

Efficiency Nova Scotia 2017 DSM Evaluation Report As part of this evaluation, the Evaluator calculated the margin of error for survey samples, free-ridership levels, in-service rates and savings adjustment ratios where applicable. The marg...

AI summary This section of the 2017 DSM Evaluation Report outlines the methods used to evaluate Efficiency Nova Scotia's programs, including calculating margins of error, conducting interviews with Retrofit consultants and participants, and performing technical reviews and on-site visits for a sample of participants.

s In the fall of 2017, a full technical review of the project documentation and on-site visits were carried out for a sample of 2017 Retrofit, New Construction and Building Optimization participants. A total of 26 Retrofit projects were re...

AI summary In 2017, a technical review of 26 Retrofit and 5 New Construction projects participated in the Custom Incentives Program was conducted, including on-site visits and interviews to validate equipment specifications and collect participant feedback.

Efficiency Nova Scotia 2017 DSM Evaluation Report The Evaluator also reviewed four Building Optimization projects by conducting a technical review of the project documentation and on-site visits so as to make recommendations on the methodo...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia (ENS) includes a technical review of Building Optimization projects, simulation model verification for New Construction projects, and a literature review to examine Effective Useful Life (EUL) values used in calculating lifetime energy savings. Free-ridership was considered negligible, and on-site visits were conducted to validate assumptions and collect participant feedback.

Efficiency Nova Scotia 2017 DSM Evaluation Report 4 RETROFIT IMPACT EVALUATION The objective of the 2017 Retrofit impact evaluation was to determine the service’s gross and net electrical energy and peak demand savings. As explained in the...

AI summary The 2017 Retrofit impact evaluation aimed to assess the energy and peak demand savings from completed and ongoing projects. The evaluation sampled 26 projects from a total of 46 completed projects, excluding those with partial savings. The sample was selected to ensure representation across various measure categories and included larger projects.

roject categories, namely compressed-air and “other”. An additional general findings section is included to discuss trends observed in the 2017 evaluation. Adjustments Made to Compressed-air Projects Of the eight compressed air projects in...

AI summary The evaluation of compressed-air projects identified adjustments needed in energy savings and peak demand calculations. Four of eight projects required changes, with adjustments typically involving operating variables or energy measurement corrections. Overall, the impact on the category was minimal.

indicated in the verification report, for which no supporting measurement or power factor value was available. These adjustments resulted in a small reduction in peak demand savings for both projects. Adjustments made to the compressed-air...

AI summary The verification report identified missing measurement data, leading to small reductions in peak demand savings. Adjustments to compressed-air projects had ratios comparable to the overall sample. The Evaluator recommends implementing a change-tracking document to update the demand-side management data system.

nning as part of a continuous process, the average demand was reasonable and representative. This was a large adjustment because the consultant had not claimed any peak demand savings for the project. The documentation of the “other retrof...

AI summary The evaluation of the Custom Incentives Program under the 2017 DSM Evaluation Report found that documentation was generally sufficient, but adjustments were needed due to discrepancies in project details and equipment changes. A change-tracking process could improve accuracy and reduce large scope changes.

Efficiency Nova Scotia 2017 DSM Evaluation Report General Findings Since the adjustment ratios for the compressed-air and other projects were not statistically different, the Evaluator calculated the ratios for all projects together. The a...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia found that the energy savings adjustment ratio was 0.995 with a 1.2% margin of error, while the peak demand savings adjustment ratio was 1.000 after a one-time adjustment. Four projects lacked sufficient M&V documentation, accounting for 19.4% of energy savings and 16.4% of peak demand savings, making accurate adjustments difficult.

usted net savings claimed over the years and (2) the true-up amount is equivalent to the total tracked project savings multiplied by the adjustment ratio determined for the year of project completion. The savings results are listed under t...

AI summary The text explains how true-up adjustments are calculated based on tracked project savings and an adjustment ratio, with examples provided for different project categories, including multi-year projects completed in 2017.

iations occurring in the levels of participation and the types of projects submitted. For 2017, the free-ridership level has been established at 9 percent, based on the results of the last five years. Since no other energy efficiency measu...

AI summary The document discusses the calculation of free-ridership and spillover levels for the Retrofit program in 2017, establishing a free-ridership level of 9% and a spillover level of 0%. The NTGR was calculated as 0.91, and net savings were determined by applying the NTGR to gross Retrofit savings. Adjustments were made to ensure net savings align with revised gross project savings.

tion projects completed in 2017. Four projects were started in 2017 and their final savings claims were all made in 2017. The fifth project was a multi-year project. 6.1.1 Project Review Findings In 2017, six projects were completed throug...

AI summary In 2017, six projects were completed through the New Construction service, with five undergoing detailed technical reviews and site visits. While energy efficiency measures were implemented as planned, savings were adjusted due to errors in energy models and subsequent calculations.

Efficiency Nova Scotia 2017 DSM Evaluation Report Overall, the revised energy savings for all five reviewed projects were 30 percent below the tracked value. As for peak demand savings, they were increased by a ratio of 3.41. To establish...

AI summary The 2017 DSM Evaluation Report found that revised energy savings for five projects were 30% below tracked values, with peak demand savings increased by a ratio of 3.41. Savings were influenced by fuel-switching, which relies on participant declarations, and greenhouse gas emission reductions were not fully accounted for in the evaluation.

tion of both the proposed and reference buildings, the interactive effects are already included. No further interactive effects factor has thus been applied to savings. 6.1.3 Effective Useful Life As part of the 2017 evaluation, the Evalua...

AI summary The document discusses the evaluation of energy savings from New Construction projects, focusing on the Effective Useful Life (EUL) of measures. The EUL values were revised based on a literature review and specific projects reviewed in 2017, with an average tracked EUL of 19.3 years for all projects generating savings that year.

etailed information regarding the process, as well as technical support for implementing energy-efficiency measures; and › Ensure more stability in staffing to allow smooth collaboration. Custom 37 Custom Incentives Program Efficiency Nova...

AI summary The 2017 Building Optimization impact evaluation assesses the gross and net electrical energy and peak demand savings from the program. Six projects were completed in 2017, with all claiming final savings. Three projects overlapped with Retrofit and were reclassified as Retrofit.

Efficiency Nova Scotia 2017 DSM Evaluation Report Project Review Findings Of the four Building Optimization projects reviewed in 2017, the Evaluator made downward adjustments to the energy savings for three of the projects and to the peak...

AI summary The 2017 DSM Evaluation Report reviewed four Building Optimization projects and found that three had energy savings adjusted downward due to incorrect engineering equations leading to energy imbalances. One project's baseline cooling tower operation hours were reduced, significantly lowering its energy savings by 37%.

were turned off, based on the information collected on site. This had a limited impact of 7 percent on this project. Overall, the adjustment ratio applied to the savings of sampled projects was 0.807. Peak demand savings were tracked for t...

AI summary The evaluation of the Custom Incentives Program under Efficiency Nova Scotia's 2017 DSM Evaluation Report indicates that adjustments were made to energy and peak demand savings based on a sample of four projects. The overall adjustment ratios were 0.807 for energy savings and 0.526 for peak demand savings, with non-reviewed projects having an adjustment ratio of 1.0.

Efficiency Nova Scotia 2017 DSM Evaluation Report The Evaluator found that the scope of projects reviewed in 2017 varied widely. While some projects included a scoping study which relied on advanced metering methods to identify deep saving...

AI summary The 2017 DSM Evaluation Report highlights variability in project scope, noting some projects focused on basic maintenance rather than full building optimization. The Evaluator adjusted basecase values to avoid overestimating savings and warned about potential free ridership. Interactive effects and effective useful life were also considered in the evaluation.

0.97 3.637 MW The NTGR was different for each component. The NTGRs listed in this table match the rounded averages obtained by dividing the net savings by the adjusted gross savings. The evaluated net savings for Custom are 8 percent below...

AI summary The document discusses the NTGR (Net-to-Gross Ratio) for the Custom Incentives Program, noting differences between evaluated and tracked net savings. The reduction in New Construction's energy savings significantly impacted the overall results, while Retrofit had minimal adjustments. The EMIS program evaluation is also mentioned, following up on 2016 recommendations.

Efficiency Nova Scotia 2017 DSM Evaluation Report 10.2 Follow-up on 2016 Evaluation Report Recommendations The EMIS component was evaluated in 2016 and recommendations were made by the Evaluator for its improvement. Table 17 below provides...

AI summary This section of the 2017 DSM Evaluation Report discusses the follow-up on recommendations from the 2016 EMIS evaluation. Three of the five recommendations were fully implemented in 2017, including promoting EMIS to new participants and encouraging integration between EMIS and SEM.

ected to be completed after the current evaluation cycle. The Evaluator then reviewed the baseline and the incremental baseline to ensure that the 2017 cumulative and incremental savings were valid. Custom 47 Custom Incentives Program Effi...

AI summary The 2017 DSM Evaluation Report discusses the validation of cumulative and incremental savings, the assumption of nil free-ridership based on 2015 participant interviews, and a literature review of EUL values used by ENS. It also outlines the EMIS Market Evaluation section.

to establish the adjusted baselines. As for the remaining two projects, the Evaluator relied on both last year’s and this year’s savings calculation workbooks to calculate actual incremental savings. While reviewing the project documentati...

AI summary The Evaluator reviewed four EMIS projects and found discrepancies between savings figures in the final tracking sheet and annual reports. The Evaluator recommends updating the tracking sheet with values from the annual reports for accuracy.

ndation, if properly implemented, will improve the results and facilitate identifying major equipment changes made between any two reporting dates. The Adjustments Made to Tracked Incremental Savings Incremental energy savings were tracked...

AI summary The document discusses adjustments made to tracked incremental savings for four EMIS projects. Projects 2 and 3 required recalculations by the Evaluator, resulting in an energy adjustment ratio of 0.89, while Projects 1 and 4 did not require adjustments as the service provider's calculations aligned with the participants’ processes.

ised compressor baseline changes made by the Evaluator. The incremental energy savings of the other three projects ranged from 23 percent to 58 percent of their cumulative energy savings respectively. Incremental peak demand savings were c...

AI summary The document discusses incremental energy and peak demand savings from various projects, highlighting the need for standardized tracking methods. Adjustments to baseline consumption data resulted in increased incremental savings, and the Evaluator recommends using 2017 data to track 2018 savings more accurately.

2017 DSM Evaluation Report Since all four 2017 participants have been continuing since 2015 or 2016, the conclusion that free- ridership is nil is still valid. The internal spillover effects were expected to be negligible because energy ma...

AI summary The 2017 DSM Evaluation Report concludes that free-ridership is nil for all participants since 2015 or 2016. Internal spillover effects are negligible due to energy management information systems measuring savings at the facility level. Net savings from the program are calculated as 1.752 GWh for energy and 0.137 MW for peak demand, with adjusted gross savings and net savings being equal due to no spillover or free-ridership effects.

Efficiency Nova Scotia 2017 DSM Evaluation Report CONCLUSION The Custom Incentives program is comprised of one well-established program component (Custom) and two relatively new program components (EMIS and SEM). The Custom Incentives prog...

AI summary The 2017 DSM Evaluation Report for Efficiency Nova Scotia shows that the Custom Incentives program achieved its peak demand savings target but fell short of its energy savings goal. The Custom program component, which includes Retrofit, New Construction, and Building Optimization, achieved 21.005 GWh in net energy savings and 3.637 MW in net peak demand savings in 2017.

. Altogether, these services generated 21.005 GWh in net energy savings and 3.637 MW in net peak demand savings at the generator in 2017, with 78 projects generating savings and 58 projects completed. The 2017 evaluation has included a tec...

AI summary The 2017 evaluation of energy efficiency services in Nova Scotia showed 21.005 GWh in net energy savings and 3.637 MW in net peak demand savings. Improvements in documentation led to a closer adjustment ratio for Retrofit services, though some projects had insufficient M&V activities. Overall, adjustments reduced ENS’s tracked value by 9 percent, with positive feedback on the Retrofit service and BDM’s support.

luated savings, and so they were the focus of this evaluation. The 2017 evaluation revealed that EMIS achieved 1.402 GWh in net energy savings and 0.255 MW in net peak demand savings at the generator. Overall, the savings were well documen...

AI summary The 2017 evaluation of EMIS showed 1.402 GWh in net energy savings and 0.255 MW in net peak demand savings. Savings were well documented but had a 20% gap due to estimation methods. Participants were satisfied with EMIS, finding it useful for real-time energy decision-making.

Sections Recommendations 1 Continue relying on BDMs to assist participants and find new markets. The 2017 evaluation showed that many participants heard about Retrofit from a BDM and those who worked with a BDM were satisfied with their in...

AI summary The evaluation recommends continuing to use BDMs to support participants and expand outreach to industrial and agricultural sectors. It also highlights the need for a more in-depth review of energy models to identify discrepancies, particularly in heat pump systems and baseline calculations, to improve accuracy in savings estimates.

2017 DSM Evaluation – Appendix Report Instrument Introduction Name: Response Company: Response Date: Response Interviewer: Response Script for Initial Phone Call or Voicemail Hi, this is from Research Into Action. I am on an evaluation tea...

AI summary This document outlines the process evaluation of Efficiency Nova Scotia’s Custom-Retrofit program, conducted by Research Into Action on behalf of ENS. The evaluation involves interviews with stakeholders to assess the program's implementation and effectiveness.

g eligibility for your customers. 1 What problems or issues, if any, have you faced? 2 What has worked well for you? 3 What problems or issues, if any, have your customers faced? Project No. 6138 6 Custom Incentives Program Efficiency Nova...

AI summary The document outlines a series of questions related to the Custom Incentives Program under Efficiency Nova Scotia, focusing on the evaluation of the 2017 DSM (Demand Side Management) initiative. It explores challenges and successes at various stages, including scoping studies, feasibility studies, implementation, and the M&V (Monitoring and Verification) phase.

nd support you need to deliver Custom rebated projects to your customers? What improvements would you like to see in ENS support? Successes, Challenges, and Opportunities Q30. How, if at all, has the Custom-Retrofit program benefited your...

AI summary The text outlines a series of questions related to the Custom-Retrofit program, focusing on its benefits, challenges, and opportunities for improvement. It also includes a table outlining research objectives and associated questions for evaluating the program's outreach, delivery, and effectiveness.

ew Construction protocol, the Evaluator considered it still appropriate for the 2017 evaluation. However, questions were added to evaluate the participant satisfaction and suggestions for improvement. Prior to conducting any site visits, t...

AI summary The 2017 DSM Evaluation involved reviewing Efficiency Nova Scotia's project documentation and conducting on-site visits to assess the Custom Incentives Program. The evaluation included participant satisfaction and suggestions for improvement.

31 Custom Incentives Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report Project No. 6138 32 Custom Incentives Program Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report Project No. 6138 33 Custom Incentives Prog...

AI summary The text includes a set of questions from the 2017 DSM Evaluation – Appendix Report related to the Custom Incentives Program by Efficiency Nova Scotia. It asks about participant experiences, including free ridership and market satisfaction with the program.

he year. Instead, the Evaluator performed a desk review made possible since mechanical zoning diagrams, drawings and pictures of the installed equipment, and simulation files were obtained from the Project No. 6138 38 Custom Incentives Pro...

AI summary The Evaluator conducted a desk review using available project documentation, including mechanical zoning diagrams and simulation files, to verify energy savings under the Custom Incentives Program. A standard questionnaire was used to validate the baseline for each energy efficiency measure, though a site visit could not be conducted for one project.

none 1.0 Energy Savings kWh 30,510 Demand Savings kW 3.483 Based on the calculations made above, the revised annual savings were decreased by 31,751 kWh/year and the demand savings were increased by 3.483 kW. Table 5: Revised Gross Savings...

AI summary The document presents revised energy and demand savings values, showing a decrease in energy savings and an increase in demand savings. It also references the 2017 DSM Evaluation recommendations by the Evaluator and includes recommendations from the 2016 evaluation that were not fully implemented.

Sections Recommendations 1 Ensure that the tracking sheet is complete and accurate. While reviewing the project documentation, the Evaluator found several discrepancies between the savings figures reported in the final tracking sheet and t...

AI summary The Evaluator identified discrepancies between the final tracking sheet and annual reports, and noted the absence of peak demand savings tracking. It also found that changes in production levels, such as increased weekend production time, affected energy savings calculations. The Evaluator recommends improving tracking and planning M&V activities to account for operational changes.

is in the future, systematic reviews of facility operational and equipment changes should be implemented by ENS to ensure the accuracy of reported savings. Project No. 6138 44 Custom Incentives Program Efficiency Nova Scotia 2017 DSM Evalu...

AI summary The 2017 DSM Evaluation involved project reviews and interviews to assess the effectiveness of the Custom Incentives Program managed by Efficiency Nova Scotia, focusing on savings persistence and participant satisfaction.

l as about savings persistence. This appendix describes the project review protocol used to conduct the reviews and the participant interviews. Project Review Protocol and Participant Interview Guide The Evaluator adapted the 2016 evaluati...

AI summary This appendix outlines the project review protocol and participant interview guide used in the 2017 DSM evaluation. It describes how the Evaluator adapted the 2016 protocol, incorporated data from previous years, and used updated reports and phone interviews to assess savings persistence and participant satisfaction.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text outlines key terms and definitions related to energy efficiency program evaluations, including gross and net savings, free-ridership, induced consumption, and interactive effects. It also describes the evaluation plan and first-year savings as critical components of assessing program effectiveness.

lement the energy Internal spillover efficiency measure introduced by a program after participating in it once, without participating in the program a second time. The proportion of incentivized products that are installed and operating. T...

AI summary The text defines various terms related to energy efficiency programs, including internal spillover, in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These terms help measure the effectiveness and impact of energy efficiency initiatives.

Efficiency Nova Scotia 2017 DSM Evaluation Report EXECUTIVE SUMMARY This report presents the results of the 2017 demand-side management (DSM) evaluation of the Direct Installation program, which is comprised of only one program component,...

AI summary The 2017 DSM Evaluation Report assesses the Direct Installation program, specifically the Small Business Energy Solutions (SBES) component. It outlines participation levels, gross and net savings for the audit and DIY paths, based on interviews, surveys, and on-site visits.

Table 1: SBES Overall Participation and Savings in 2017 Participation Level Gross Savings NTGR Net Savings Audit Path Energy Savings 1.519 GWh 0.93 1.415 GWh Lifetime Energy Savings 50 projects 20.178 GWh 0.93 18.796 GWh Peak Demand Saving...

AI summary The table presents data on the Small Business Energy Solutions (SBES) program's participation and energy savings in 2017, including gross and net savings by audit and DIY paths, and overall totals. The report is part of the 2017 DSM Evaluation Report by Efficiency Nova Scotia.

its inception in 2015,2 and clearly illustrates SBES’s steady increase in participation level and the associated savings. Figure 1: Overview of SBES Performance, 2015-2017 Number of Projects Net Savings 417 7.887 259 4.120 0.025 0.016 0.01...

AI summary The document highlights the success of the Small Business Energy Solutions (SBES) program from 2015 to 2017, showing an increase in participation and energy savings. Participants expressed high satisfaction with the program, particularly with the equipment and audit services, though a few had concerns about incentives and energy savings.

m of the auditor and usefulness of the audit. The other few participants who were less satisfied indicated dissatisfaction with the amount of the incentive (n=2) and the energy savings achieved (n=2). When asked to rate the predetermined f...

AI summary The evaluation of the Small Business Energy Solutions (SBES) program found that participants were generally satisfied with the audit process but had concerns about incentive amounts and energy savings. Financial challenges and lack of knowledge were cited as main barriers. Suggestions for improvement included better promotion, improved communication, and increasing rebate amounts.

Efficiency Nova Scotia 2017 DSM Evaluation Report SBES Energy Savings The Evaluator reviewed ENS’s gross savings estimates by conducting a desk review using the information contained in the tracking system, the CIRx Screening Tool and the...

AI summary The 2017 DSM Evaluation Report reviewed Efficiency Nova Scotia's energy savings estimates, identifying discrepancies due to mismatched operating conditions. Adjustment ratios for energy and peak demand savings were calculated, and a net-to-gross ratio was determined after assessing free-ridership and spillover effects from the SBES program.

the DIY path, and the spillover was found to be nil. Table 2 below presents the NTGR values calculated as part of this evaluation. Table 2: Net-to-gross Ratio Audit Path DIY Path Overall Value Free-ridership Level 7% 13% 12% Spillover Leve...

AI summary The 2017 DSM Evaluation Report by Efficiency Nova Scotia compares the net-to-gross ratio (NTGR) for the audit and DIY paths, showing a free-ridership level of 7% and 13% respectively, with no spillover. The report also compares tracked and evaluated savings for the Small Business Energy Solutions (SBES) program, highlighting discrepancies between initial gross savings and net savings after adjustment.

Efficiency Nova Scotia 2017 DSM Evaluation Report The evaluated net energy savings are 10 percent below the corresponding tracked figure. This is mainly due to the adjustment made to the gross savings, since the 2017 evaluated NTGR was sli...

AI summary The 2017 DSM Evaluation Report found that evaluated net energy savings were 10% below tracked figures, primarily due to adjustments in gross savings and discrepancies in lighting measures. Evaluated net peak demand savings were 24% lower than tracked values, mainly due to revised peak coincidence factors for lighting measures. SBES performed well with increased participation and satisfaction, and the Evaluator provided recommendations for improvement.

moving forward on the right track and further improve its performance and effectiveness, the Evaluator has the following recommendations to make. These recommendations can also be found in Appendix I. 2017 SBES-R1. Implement an internal pr...

AI summary The Evaluator recommends that Efficiency Nova Scotia (ENS) implement an internal procedure to thoroughly review complex custom projects to ensure accurate savings calculations. This is necessary due to the specialized technical knowledge required for validating savings from such projects, especially since they often represent the largest savings.

fficiency codes and standards introduced in Nova Scotia. To carry out this assignment, Econoler collaborated with Research Into Action, Equilibrium Engineering and Corporate Research Associates (CRA). Econoler served as team leader in the...

AI summary The document describes an evaluation of energy efficiency programs in Nova Scotia conducted in 2017 by Econoler, Research Into Action, Equilibrium Engineering, and CRA. The evaluation covered various programs, including their components and types of evaluations performed, with a reduction in scope compared to 2016.

on performed for each program component. 3 On January 1, 2015, ENS became a franchise owned by the Province of Nova Scotia and was licensed to the new not-for- profit organization EfficiencyOne. Direct Installation Program 1 Direct Install...

AI summary The document discusses the transition of Efficiency Nova Scotia (ENS) into a franchise owned by the Province of Nova Scotia and licensed to EfficiencyOne. It also references a 2017 DSM Evaluation Report, which includes a table outlining the types of evaluations conducted for each program component in 2016 and 2017.

2017 DSM Evaluation Report recommended changes and as a result, 10 projects were completed for two pilot participants and more projects could be be completed in 2018. 1.2 Follow-up on 2016 Evaluation Report Recommendations SBES was evaluat...

AI summary The 2017 DSM Evaluation Report discusses the follow-up on recommendations from the 2016 evaluation of the Small Business Energy Solutions (SBES) program. Several recommendations have been implemented, while one is still being implemented.

Efficiency Nova Scotia 2017 DSM Evaluation Report Other improvements and modifications were also brought to this program component this year in order to facilitate the auditors’ work and better align their qualifications with the facilitie...

AI summary Efficiency Nova Scotia (ENS) made several improvements to its Demand Side Management (DSM) program in 2017, including developing cost estimates for lighting measures, categorizing auditors into tiers, and revising application forms to improve savings calculations for DIY projects. These changes were implemented based on recommendations from the 2016 SBES report.

2018. The Evaluator expects the impact of this change to show in the 2018 impact evaluation, since new forms had not yet been made available to the participants visited as part of the 2017 evaluation. 1.3 Participation History SBES was ini...

AI summary The SBES program, launched in 2015, saw a significant increase in participation and energy savings between 2016 and 2017, with a 79% increase in gross energy savings and 61% increase in the number of projects. In 2017, 417 projects were completed, including 50 audit-path and 367 DIY-path projects.

OGY This section describes the methodology used and the activities carried out for the SBES 2017 evaluation. Figure 6 below shows the various data-collection activities carried out for the evaluation. Figure 6: Methodological Model The Eva...

AI summary This section outlines the methodology for the 2017 evaluation of the Small Business Energy Solutions (SBES) program. It describes data collection activities, the use of a 10% margin of error at a 90% confidence level, and the process of reviewing program documentation and meeting with the program manager to assess implementation.

he margin of error indicates the relative level of precision4 of the measurement, while its associated confidence level indicates the probability of a measurement falling within this margin of error.5 As part of this evaluation, the Evalua...

AI summary This section discusses the evaluation of the SBES program, including survey results and project reviews. A participant survey was conducted with 70 SBES participants, yielding an 8.5% margin of error at a 90% confidence level. Project reviews were carried out by Equilibrium Engineering in October and November 2017.

Efficiency Nova Scotia 2017 DSM Evaluation Report A total of 50 SBES projects were reviewed, including 35 DIY path projects and 15 audit path projects. These project reviews and visits were conducted to validate whether the measures implem...

AI summary The 2017 DSM Evaluation Report reviewed 50 SBES projects, including on-site visits and validation of energy efficiency measures. Adjustments to gross savings were made based on findings, and an effective useful life review was conducted using technical references to evaluate ENS's calculations.

ues used by ENS to calculate lifetime energy savings, the Evaluator conducted a thorough literature review by consulting technical reference manuals (TRMs), evaluation reports and relevant studies. Small Business Energy Solutions 11 Direct...

AI summary The 2017 SBES market evaluation found high participant satisfaction with the Small Business Energy Solutions program, with an average satisfaction rating of 8.8 out of 10. Participants were particularly satisfied with the equipment installed through the program, rating it 9.4 out of 10. Some dissatisfaction was noted regarding contractors, incentive amounts, and energy savings.

in duty is to identify potential energy-saving opportunities, their failure to notice the air compressors installed and used on site, means that additional energy-saving opportunities were overlooked. Overall, all SBEAs conducted their aud...

AI summary The evaluation report highlights that while SBEAs conducted audits professionally, they missed identifying air compressors on site, leading to overlooked energy-saving opportunities. The report suggests implementing a checklist during initial interviews to ensure all equipment is documented.

The Evaluator reviewed ENS’s gross savings estimates by conducting a desk review using the information contained in the tracking system, the CIRx Screening Tool and the findings of the on-site visits. In 2017, the SBES tracking sheet recor...

AI summary The Evaluator assessed ENS’s gross savings estimates using a desk review and on-site visits. In 2017, 1,823 measures were implemented across 10 categories, with lighting measures making up 94% of the total. A stratified sampling approach was used due to lighting's dominance in savings. Adjustments to baseline wattage, efficient wattage, and annual hours of use were noted.

Scotia 2017 DSM Evaluation Report Table 10: Average Free-ridership Level by Path Average Free- Sample Population Margin of Percentage of Gross Energy Project Path ridership Level Size Size12 Error Savings in SBES Population Audit 7% 20 49...

AI summary The 2017 DSM Evaluation Report discusses free-ridership levels, internal spillover, and NTGR calculations for SBES. It found no internal spillover and calculated NTGR values of 0.93 and 0.87 for the audit and DIY paths, respectively. Net savings were determined using the NTGR formula.

0.000 0.001 0.001 0.000 1.041 at the Meter (MW) Line Loss Factor 1.071 1.071 1.071 1.071 1.071 1.071 1.071 1.071 1.071 1.071 - Net Peak Demand Savings – 0.939 0.139 0.000 0.000 0.036 0.000 0.000 0.001 0.001 0.000 1.116 at the Generator (MW...

AI summary Table 13 compares tracked and evaluated energy and peak demand savings for the Small Business Energy Solutions Direct Installation Program by Efficiency Nova Scotia. The table shows discrepancies between the tracked savings and the evaluation results for both energy and peak demand savings.

Efficiency Nova Scotia 2017 DSM Evaluation Report CONCLUSIONS SBES was launched in mid-2015 to replace BES, so 2017 marked SBES’s second full year of operation. Even though participation nearly doubled and reached a new high, it was not su...

AI summary SBES achieved 7.887 GWh in total net energy savings and 1.363 MW in peak demand savings in 2017, falling short of its 8.8 GWh and 1.9 MW targets. Participation nearly doubled, and satisfaction was high, but non-lighting measures have untapped potential for future savings.

ave quite some untapped potential for achieving future energy and peak demand savings through SBES and therefore suggests looking for new ways to encourage the implementation of non-lighting measures. The Evaluator also found that there wa...

AI summary The text discusses the potential for energy and peak demand savings through the Southwestern Nova Scotia Energy Services (SBES) program and highlights the need for improved tracking of savings, particularly from complex custom projects. It also mentions efforts by ENS to improve data collection and monitoring of program performance.

ES: Effective Useful Life Appendix VII - SBES: Algorithm for Free-ridership Calculation Appendix VIII - SBES: Algorithm for Internal Spillover Calculation Appendix IX - SBES: Calculation Example Direct Installation Program 32 160 Saint-Pau...

AI summary This document provides appendices related to the Southwestern Nova Scotia Energy Services (SBES) algorithms for calculating free-ridership and internal spillover, as well as an example of the calculation. It is part of the 2017 Demand Side Management (DSM) evaluation report by Efficiency Nova Scotia.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report 2017 RECOMMENDATIONS This appendix summarizes all the recommendations made by the Evaluator as part of the 2017 evaluation, as well as all 2016 evaluation recommendations that we...

AI summary The 2017 DSM Evaluation recommends that Efficiency Nova Scotia (ENS) implement an internal procedure to thoroughly review complex custom projects to ensure accurate savings calculations. This is necessary due to the specialized technical knowledge required for validating savings from such projects, particularly those based on energy models or complex engineering equations.

2017 DSM Evaluation – Appendix Report S3. Do you have any recommendations for improving the Small Business Energy Solutions program? Anything else? Participant Suggestions for SBES 2016 2017 Sample Size 37 70 Better marketing/awareness 5%...

AI summary The 2017 DSM Evaluation – Appendix Report discusses participant feedback on the Small Business Energy Solutions (SBES) program, highlighting areas for improvement such as better marketing, improved communication, increased rebates, and simplifying the program. The report also includes an audit path question about pre-installation planning for energy-efficient upgrades.

by participating in that program, a company representative took into account the cost-effectiveness of energy efficient upgrades when evaluating different options for the facility. 2016 2017 Impact of Previous ENS Program Component Partici...

AI summary The text discusses the impact of previous participation in ENS programs and the awareness of energy efficiency promotional materials. It highlights that a majority of respondents agreed on the cost-effectiveness of energy-efficient upgrades and that most had previously seen ENS promotional materials.

FR1 = 0% PA2 Score: FR1a FR4. Level of influence – 7 factors (Scale 0 to 10) FR4 = MAX(a; b; c; d; e; f; g) PA3 Score: (10 − FR4) x 10% FR3b. [ASK IF TOTAL MEASURE QTY>1] If the audit, rebate and/or financing had not been offered, what is...

AI summary The text outlines a scoring framework used in the evaluation of the Direct Installation Program under Efficiency Nova Scotia's 2017 DSM Evaluation. It includes questions assessing the influence of the program on participants' decisions to install energy-efficient upgrades and their likelihood of postponing such upgrades.

Efficiency Nova Scotia 2017 DSM Evaluation – Appendix Report IF PA4 < MEAN (PA1; PA2; PA3): Free-Ridership MEAN (PA1; PA2; PA3;PA4) OTHERWISE MEAN (PA1; PA2; PA3) CI2/4a. [The company’s previous participation in an ENS program]/[The energy...

AI summary The document discusses the evaluation of Efficiency Nova Scotia's 2017 Demand Side Management (DSM) program, focusing on factors influencing participants' decisions to install energy-efficient upgrades, including previous company participation and promotional materials from ENS.

istributed by ENS] prompted a company representative to take into account the cost-effectiveness of energy efficient upgrades when evaluating different options for the facility. (Agree or Disagree) IF CI = 0 OR 1: Revised FR = FR Revised F...

AI summary The text discusses the calculation of internal spillover from the SBES program by using participant surveys to assess the implementation of additional energy efficiency measures and their savings. It outlines how the level of influence from the program is used to attribute savings.

pant’s decision to implement these additional measures. The value established regarding this level of influence was used to determine the portion of the additional savings attributable to the program. Then, the level of internal spillover...

AI summary The text outlines a methodology to calculate internal spillover by comparing additional savings attributable to a program with total program savings. It also provides an example calculation for revising energy and peak demand savings, focusing on a lighting measure category that accounted for a significant portion of savings in the SBES program in 2017.

Gross and net savings calculated by the Evaluator using the parameters (unitary savings values, installation rates, interactive effects, net-to-gross Evaluated savings ratio, etc.) validated or measured during the evaluation process. Unles...

AI summary The text defines various terms related to energy efficiency program evaluation, including gross and net savings, free-ridership, induced consumption, and interactive effects. It outlines the evaluation plan and first-year savings, emphasizing the importance of measuring and validating energy efficiency program outcomes.

without participating in the program a second time. The proportion of incentivized products that are installed and operating. This rate is usually applied to programs through which rebates are offered in-store In-service rate and products...

AI summary The text defines several metrics used in energy efficiency programs, including in-service rate, lifetime energy savings, line loss factor, margin of error, market effects, net energy savings, and net-to-gross ratio. These metrics are essential for evaluating the effectiveness and impact of energy efficiency initiatives.

Efficiency Nova Scotia 2017 DSM Evaluation Report To determine the energy savings value of an EnerGuide rating point, the Evaluator used the unitary savings obtained from the 2015 evaluation, which was based on the evaluation of ENS’s New...

AI summary This report evaluates the energy savings from EnerGuide ratings based on 2015 data from New Home Construction participants. Building simulations overestimated electricity consumption by 16%, leading to an adjusted average savings of 1,278 kWh/year per EnerGuide point.

Marketing Evaluation 2016/17 Final Report November 28, 2017 Prepared by 3934 NE Martin Luther King Jr. Blvd., Suite 300 Portland, Oregon 97212 www.researchintoaction.com Final Report Marketing Evaluation November 28, 2017 Prepared For: Eff...

AI summary This document is the final report of a marketing evaluation conducted for Efficiency Nova Scotia (ENS) in 2016/17 by Research Into Action, Inc. and Corporate Research Associates, Inc. The report outlines the structure, including an executive summary, introduction, and logic model analysis to assess ENS's marketing initiatives.

................................................................. 2 2.3. ENS’s Marketing Logic Model and Indicator Tables ....................................................................... 3 3. Marketing Actor Interviews ................

AI summary The document outlines a structured analysis of Efficiency Nova Scotia's (ENS) marketing strategies, including logic models, stakeholder interviews, and best practices assessments. It evaluates marketing effectiveness, team evolution, and future opportunities, with appendices detailing logic models and indicator tables for integrated communications tools.

program planning and implementation efforts, so that marketing and audience insights and concerns are factored early on and throughout the design process for new programs and services. Best Practices Assessment of New Marketing Materials E...

AI summary ENS's program planning emphasizes incorporating marketing insights and audience concerns into service design. Research Into Action evaluated ENS's new marketing materials against best practices, assessing campaign effectiveness through a systematic review of branding strategies and behavioral theory applications.

Executive Summary Page V Marketing Evaluation 1. Introduction In late 2015 and continuing into 2016, Efficiency Nova Scotia (ENS)3 executed a major refresh and revamping of its marketing and communications structure, communications, goals,...

AI summary Efficiency Nova Scotia (ENS) revamped its marketing strategy in 2015-2016, consolidating teams, hiring new leadership, and partnering with a creative agency. Research Into Action evaluated the new approach, assessing logic models, operational effectiveness, and adherence to best practices through stakeholder interviews and campaign analysis.

team and refreshed marketing efforts. Traditionally, a logic model illustrates the resources available to a “program” and the linkages between what the program does and what it strives to accomplish. To adapt the logic model framework to t...

AI summary The document describes the adaptation of a logic model to evaluate Efficiency Nova Scotia's (ENS) marketing efforts, including internal infrastructure and external campaigns. The model aids in setting goals, strategies, and assessing effectiveness over time, while acknowledging it is a dynamic tool that evolves with ENS's objectives.

’s marketing team—and the team’s goals and objectives—evolve. 2.2. Logic Model Terminology and Structure Logic models have four key components, which we have defined as follows for this effort:  Inputs are the resources dedicated to or us...

AI summary The text outlines a logic model for a marketing team, defining four components: inputs (resources and challenges), activities (team actions), outputs (direct results), and outcomes (desired changes). Examples include brand consistency, marketing campaigns, and infrastructure development tied to Efficiency Nova Scotia (ENS) programs.

utcome could be having a fully staffed marketing team, and a short-term campaign-related outcome could be that all new marketing materials adhere to refreshed brand guidelines for ENS. Medium-term outcomes take longer to realize. For examp...

AI summary The text outlines short-term, medium-term, and long-term outcomes for ENS marketing activities, emphasizing measurable indicators like staff understanding, campaign impressions, and program participation. It highlights the development of an indicator table to assess marketing effectiveness and progress toward organizational goals.

que pageviews, and online conversions for a particular target behavior (e.g., digital media impressions in relation to website traffic and/or specific target behaviors, such as application downloads). The indicator tables identify potentia...

AI summary The text outlines ENS's marketing logic model and indicator tables from its 2017 strategic plan, focusing on goals to increase program participation, enhance customer experience, and improve marketing effectiveness. It highlights resources like staff expertise and funding, and notes challenges in maintaining brand consistency during the ENS brand refresh.

engagement • Streamlined delivery across • Streamlined measurement • Increased customer trust in Long-Term Outcomes channels and platforms and reporting of marketing ENS • New opportunities quickly activities • Customers participate in ide...

AI summary The text outlines Efficiency Nova Scotia's (ENS) marketing strategies, emphasizing streamlined delivery, customer trust, and stakeholder engagement. It highlights long-term outcomes such as increased participation in programs, brand consistency, and customer referrals. The document also references Key Performance Indicators (KPIs) and includes appendices with detailed indicator tables for monitoring marketing activities.

standing of roles and responsibilities; adequate resources and support for marketing staff; and effective communication among marketing team members and between the marketing team and other ENS staff. Indicators: Marketing staff and progra...

AI summary The text outlines ENS's focus on ensuring marketing staff understand their roles, have adequate resources, and maintain effective communication. It emphasizes indicators like staff adherence to brand guidelines, workload management, and data collection methods such as IDIs and Google Analytics to evaluate program implementation effectiveness.

Marketing plan and campaign materials reviews; IDIs with ENS marketing staff and program managers; review of Google Analytics metrics; marketing tracking system and program databases. ENS’s long-term expectations for this activity are the...

AI summary ENS evaluates marketing strategies, emphasizing integrated delivery, brand consistency, and responsive campaign adjustments. Key indicators include coordinated materials and staff effectiveness. Tracking systems like Basecamp are used for campaign management, with data collected via IDIs and analytics reviews.

mpressions, cost-per-click, etc.); and conversion rates, which look at online traffic and engagement associated with specific target behaviors (e.g., clicks, downloads, application submissions, etc.). Indicators: Written guidelines for pro...

AI summary The text outlines marketing evaluation methods, focusing on digital analytics, conversion rates, and user feedback mechanisms. It emphasizes the importance of user-friendly tracking systems, regular staff engagement, and satisfaction with processes, supported by in-depth interviews and digital metrics.

Data sources and data collection methods: IDIs with marketing staff, program managers, and system support/maintenance staff; periodic review of summary and campaign-specific reports. 2.3.3. Deliver Marketing Campaigns Outputs from ENS’s de...

AI summary The text outlines ENS's marketing campaign delivery processes, including creative outputs, media buys, and KPI tracking. It emphasizes brand consistency, customer awareness, and stakeholder engagement, with data collected via IDIs and campaign reports. Short- and long-term goals focus on monitoring KPIs and aligning messaging with brand guidelines.

ogram-related interactions with customers), customers perceive ENS as a trusted advisor. ENS also hopes participants will share their program experiences with family and friends and refer them to ENS. Indicators: Number and percent of targ...

AI summary ENS aims to enhance customer trust and program participation through marketing campaigns and exceptional service. Key indicators include participation rates, perceived trust in ENS, and trade ally engagement. Data sources include surveys, interviews, and program databases to evaluate marketing effectiveness and stakeholder perceptions.

sources of program information; changes in the number and level of trade ally participation in programs and trainings; participant, nonparticipant, and trade ally satisfaction with ENS. Data sources and data collection methods: Program dat...

AI summary The text outlines data collection methods for evaluating ENS programs, including surveys and interviews with participants, nonparticipants, and trade allies. It also discusses the 2016 restructuring of ENS's marketing and communications team, including merging departments, hiring a new manager, and unifying program and corporate marketing efforts.

ated team and lead efforts to unify program and corporate marketing, to find “synergies and opportunities to leverage the two teams together,” and to clarify processes and roles; and  Engaged a new creative agency to refresh the ENS brand...

AI summary Efficiency Nova Scotia (ENS) unified its marketing and communications teams to create a more integrated approach, refresh its brand platform with a new empowering message, and shift from transactional to behavior-changing energy efficiency campaigns. This followed interviews conducted by Research Into Action to evaluate marketing strategies.

and collaboration between the teams. They described the marketing team, and the combined department, as increasingly cohesive and collaborative, where individual members are recognized and supported. In terms of ENS’s agency of record and...

AI summary The text highlights effective collaboration between ENS's marketing and communications teams, the successful partnership with ENS's agency and media partner, and the positive impact of the new marketing strategy on organizational clarity. Respondents suggest improvements in interdepartmental collaboration, role clarity, cross-sector marketing integration, and small business targeting strategies.

esses:  Foster more communication about marketing goals and activities for individual programs across departments, particularly between the marketing team members and program managers.  Ensure all marketing actors are well schooled with...

AI summary The text highlights recommendations to improve internal communication between marketing teams and program managers, enhance tool usage training (e.g., Basecamp), and address staff perceptions about collaboration with the agency. It emphasizes the need for better alignment on marketing goals and resolving challenges in using digital tools for document management.

• Marketing and program databases • Marketing and program tracking show target populations reached • Marketing and program staff have mechanisms to identify underserved populations/markets Indicator Table—Enhance/Develop Integrated Marketi...

AI summary The text discusses the implementation of marketing plans and the use of marketing evaluation tools, emphasizing the clarity of the plan, effective implementation by staff, and the use of data sources such as IDIs and Google Analytics metrics to monitor progress.

Indicator Table—Enhance/Develop Marketing Communications Tracking Systems Page C-1 Marketing Evaluation Short-Term Outcomes Indicators Data Sources & Collection Approaches 1. Relevant staff find tracking • Tracking systems allow easy data...

AI summary This document outlines the short-term outcomes and indicators for enhancing and developing marketing communications tracking systems. It emphasizes the importance of user-friendly systems, regular usage by staff, and the ability of systems to capture and store necessary information for marketing efforts.

Indicator Table—Deliver Marketing Campaigns Page D-3 Marketing Evaluation Medium-Term Outcomes Indicators Data Sources & Collection Approaches ENS programs received informally, including at program events) 8. Stakeholders increasingly • Ch...

AI summary The document outlines indicators for evaluating the engagement of stakeholders in ENS programs, including participation rates, training attendance, and project involvement by trade allies, with data collected through program databases, surveys, and interviews.

over time participant trade allies • Changes in nonparticipant trade ally awareness of program over time • Surveys of nonparticipant trade allies Long-Term Outcomes Indicators Data Sources & Collection Approaches 1. Customers have increasi...

AI summary The text outlines long-term outcomes and indicators for evaluating customer trust and participation in ENS programs. It highlights the importance of tracking changes in trust levels among participants and nonparticipants, as well as measuring program participation and energy savings through surveys, interviews, and database analysis.

• Surveys of participants • Short interviews with BDMs 3. Customers continue to share • Number and percent of participants reporting they learned of program • Surveys of participants experiences and refer friends through family/friends • I...

AI summary The text outlines methods used to evaluate the effectiveness of ENS programs, including surveys, interviews with Business Development Managers (BDMs), and data from program databases. It highlights customer engagement, stakeholder participation, and the involvement of trade allies in the programs.

Table of Contents Page I Practices and Procedures Evaluation Volume 1: Incentive Setting 1. Introduction This evaluation report presents findings from an initial process assessment of the incentive setting process at Efficiency Nova Scotia...

AI summary This report evaluates the incentive setting process at Efficiency Nova Scotia (ENS) based on the CLEAResult Incentive Setting Methodology and EfficiencyOne’s Implementation Plan. ENS submitted these documents in 2016 and revised them in 2017, which were accepted by the Nova Scotia Utility and Review Board (UARB) in July 2017. Research Into Action was contracted to conduct an independent review as part of the 2017 Evaluation and Verification process.

ngs related to staff experience using the Incentive Setting Workbook, the broader decision process around incentive setting, and ENS compliance with the CLEAResult report and the Implementation Plan. 1.1. Incentive Setting Process Overview...

AI summary The document outlines ENS's process for setting and reviewing energy efficiency program incentives, including the development of an Incentive Setting Manual and Workbook following the CLEAResult report and Implementation Plan. Incentives are reviewed annually and for measures exceeding spending thresholds.

t for total incentive spending will receive a second review in the fourth quarter of the year.1 All new measures must also go through the incentive setting process before being added to the portfolio. ENS implemented the incentive setting...

AI summary ENS implemented an incentive setting process in 2017 to review existing and new energy efficiency measures. The process involves program managers and an internal reviewer, with plans to scale it to all incentives by Q2 2018. Financial simulations and market research are also being conducted to support the incentive setting process.

forts specified in its 2017-2018 Market Research Plan to gather data to inform the incentive setting process. We describe the financial simulations and market research efforts further in Section 4.5. 1.2. Evaluation Objectives This evaluat...

AI summary This section outlines the evaluation objectives for the incentive setting process conducted in Q3 and Q4 2017, focusing on the experience of program managers and team members, the effectiveness of the process, and opportunities for improvement to scale it in 2018.

Introduction Page 2 Practices and Procedures Evaluation Volume 1: Incentive Setting • What elements of the incentive setting process are most time-consuming for program team members? • How long does it take for program team members to comp...

AI summary The text outlines a set of questions aimed at evaluating the incentive setting process within ENS, focusing on time consumption, experience impact, compatibility with different measures, information sufficiency, and compliance with the Implementation Plan.

iously collected as inputs to their incentive-setting decisions? What effect do these differences have on the previous incentive levels?  Compliance with the Implementation Plan • How well does the new process comply with five specific ar...

AI summary The evaluation approach involves reviewing documents, analyzing workbook feedback from program staff, and conducting interviews to assess compliance with the Implementation Plan and the effectiveness of incentive-setting processes.

Introduction Page 3 Practices and Procedures Evaluation Volume 1: Incentive Setting  In-depth interviews with Program Managers and other ENS staff. The evaluation team conducted hour-long interviews with all five program staff members who...

AI summary This section evaluates the experience of program staff in completing the Incentive Setting Workbook, based on their written feedback and in-depth interviews. It examines the time required, challenges faced, and resources needed to improve the process.

ok and through in-depth interviews. It explores the time required to complete the Workbook, the challenges staff faced in completing the Workbook, and the resources needed to make the process easier. 2.1. Time Required to Complete the Work...

AI summary The document discusses the time and resource challenges faced by program staff in completing the Workbook for reviewing incentive measures. Staff reported that the process is time-consuming, often taking multiple days per measure, and that the current system is less efficient than previous methods, creating concerns about scalability and workload.

instance of the Workbook for each tier. Finally, two staff noted that their program components use multiple program delivery models for some measures, but the Workbook allows them to select only one. 2.3. Suggestions for Further Resources...

AI summary Program staff highlighted challenges with the Workbook, noting that it only allows selection of one program delivery model despite using multiple models for some measures. They also suggested improvements such as a central reference source, better prioritization of fields, and additional support from ENS regulatory staff to streamline the incentive setting or review process.

from contacting the same market actors, staff noted that coordination could help to avoid placing too great a burden on market actors involved in evaluation, measurement, and verification activities. 2.3.2. Prioritization of Workbook Input...

AI summary The text discusses the need for coordination to reduce burdens on market actors involved in evaluation, measurement, and verification. It also highlights the importance of prioritizing Workbook inputs based on the nature of each measure and the need for increased regulatory staff support to help program staff interpret Workbook data effectively.

heir incentive setting decisions than they had in the past. Specifically, two staff reported that conversations with contractors or distributors provided valuable insight to their incentive decisions. The respondents reported the informati...

AI summary The document discusses how program staff use the 'Inputs & Calcs' tab in the Workbook to make decisions about incentive levels, emphasizing its importance in determining cost effectiveness and suggesting streamlining the review process by focusing on this tab.

Verify that ENS created a Consolidated Calculator and document the Objective extent to which it is being used in the incentive setting process. The Consolidated Calculator is integrated into the Workbook (on tab titled “Inputs & Calcs”), w...

AI summary The document requests verification that ENS created a Consolidated Calculator and its use in incentive setting. It also asks to verify and document financial simulations conducted by ENS for existing incentives, with recommendations to expand simulations to all program components and evaluate against cost, budget, and cost-effectiveness thresholds.

all program components, and check the most current incentives against the following screening thresholds:  Cost to participants  Program budget  Cost effectiveness 3 Prescriptive measures represent those measures whose energy and demand...

AI summary ENS conducted financial simulations in June 2017 to evaluate the appropriateness of existing incentives and establish a baseline for future reviews. The simulations were more effective for some measures and program components than others, leading to lessons learned that can guide future efforts.

found that the simulations worked better for some measures and program components than others. Below are descriptions of lessons learned that can be used to inform future financial simulation efforts:  The thresholds work best for measure...

AI summary The simulations for financial incentives showed varying effectiveness depending on the measure and program component. Prescriptive measures worked best with thresholds, while complex components like whole home programs required different modeling approaches. Custom incentives and caps were also discussed as important considerations for cost-effectiveness and over-spending prevention.

to provide it? Q13. Of all the information in the incentive setting workbook, what, if anything, did you not find helpful in setting an appropriate incentive level? Why wasn’t it helpful? Q14. How, if at all, does the information you need...

AI summary The document contains a series of questions aimed at evaluating the incentive-setting process within Energy Efficiency Nova Scotia (ENS). It explores the effectiveness of the new process, differences in information needs across measures, and the impact of the process on market engagement and defensibility of incentive levels.

1 Unsuccessful attempt 8 Total 12 To minimize the burden on Retrofit service participants, the BD evaluation team coordinated with the Research Into Action team conducting the Retrofit service process evaluation, and developed a single ins...

AI summary The document discusses the evaluation of the Retrofit service and BER participant interview guides used to assess the effectiveness of Business Development Managers (BDMs) in promoting ENS’s energy-efficiency programs. The guides were developed to evaluate internal collaboration and BDMs’ outreach and sales efforts to BNI customers.

] Q25. Do you typically have one key contact at each customer’s organization, or do you work with more than one individual? How does this vary for local versus international organizations? Q26. Is/are the key contact/s typically in decisio...

AI summary The text consists of a series of questions posed to Business Development Managers regarding customer interactions, data management, and challenges and successes of the BD team. The questions focus on customer engagement strategies, key contacts, support needs, customer satisfaction, and the impact of new systems on communication and workflow.

you get projects you may not have otherwise gotten? How so? [Probe for influence of BDMs] Q28. How satisfied have you been with the program in terms of the requirements asked of you? Why? Q29. Overall, how satisfied have you been with Effi...

AI summary The document contains interview questions and an introduction for participants in the Custom-Retrofit program evaluation. It focuses on gathering feedback on program effectiveness, challenges, and opportunities for improvement, with a particular emphasis on the role of Business Development Managers and Efficiency Nova Scotia's support.

CA will be provided by EfficiencyOne to NS Power. It incorporates the requirements of the Act, with the exception of the term of one year in accordance with the EPIA, which provide that the Agreement: (a) Not be terminable or terminated un...

AI summary EfficiencyOne will provide a contract to NS Power under the EPIA, with terms excluding a one-year termination clause. The 2019 DSM targets align with the 2016-2018 plan, incorporating quarterly performance reporting on energy savings and spending. Pilots and stakeholder collaboration are emphasized in the 2019 plan.

ion by EfficiencyOne contrary to the terms of 20 this Agreement or the Act, or in consequence of EfficiencyOne's contravention of CASL. 21 12. PERFORMANCE REQUIREMENTS AND EVALUATIONS 22 12.1 EfficiencyOne's performance under the terms of...

AI summary The text outlines contractual obligations between EfficiencyOne and NS Power under the Electricity Efficiency and Conservation Act, including performance evaluation requirements set by the UARB, force majeure provisions, and compliance with CASL. It emphasizes adherence to legal standards and mitigation of disruptions.

47 Performance Requirements 48 I. UARB-APPROVED PERFORMANCE TARGETS, THRESHOLDS, AND 49 INDICATORS 50 51 a) Performance Targets and Thresholds: 52 i. EfficiencyOne is deemed to be in substantial compliance with the UARB- 53 approved Plan P...

AI summary The document outlines UARB-approved performance targets, thresholds, and indicators for EfficiencyOne. It specifies that 90% achievement on two key performance targets—incremental annual net energy savings and peak demand savings—is required for substantial compliance. If targets are not met, a regulatory process will be initiated. Additional indicators include lifetime savings, ratepayer benefits, spending, customer satisfaction, and low-income program metrics.

20.2 0,268 '" 0.021 3 Currency 1s expressed m 2019 dollars. 4 Columns may not add correctly, due to rounding. 5 n Based on values of program Weighted Average Measure Lives, which have been derived from the 2017 Evaluation process. 6 1, Inc...

AI summary The 2019 program targets are based on the approved 2016-2018 Plan and recent evaluation results. The total budget and savings targets for 2019 are similar to the average annual targets from the previous plan, with a slight increase due to the allocation of $1 million to Enabling Strategies for research and pilots focused on demand reduction. The overall unit cost has increased to diversify the measure mix and reduce reliance on lighting savings.

1 2.2 Treatment of Rate and Bill Impact Analysis 2 3 ENS has conducted a forward-looking Rate and Bill Impact Analysis (RBIA) for the 2019 4 DSM Resource Plan as it normally would with a new multi-year Plan filing. The analysis 5 shows tha...

AI summary ENS has conducted a Rate and Bill Impact Analysis (RBIA) for the 2019 DSM Resource Plan, showing that expected rate and bill effects are in line with previously approved DSM activities. The planned DSM spending is less than DSM cost recovery in 2019 rates, meaning no incremental costs will need recovery after the Rate Stabilization Period. A historical RBIA will also be filed in October 2019, and ENS will continue working with the DSM Advisory Group and Synapse to improve the model.

ional circumstances in which an application to the UARB must be made; and 23 • Additional indicators such as Total Resource Benefits and Pipeline of Projects 24 (present in the initial recommendations from Dunsky Energy Consulting) are not...

AI summary The document discusses the exclusion of certain indicators like Total Resource Benefits and Pipeline of Projects from the application process due to ongoing discussions about customer non-energy benefits and their role in the TRC Test. It also highlights improvements in ENS’s ability to track lifetime energy savings and report ex-post results annually.

1 • Program impact and process evaluations; 2 o Evaluated savings results; 3 o Delivery Agent input; and 4 o Participant feedback. 5 6 EfficiencyOne’s objective in using this approach is to achieve cost-effective and affordable 7 energy an...

AI summary EfficiencyOne aims to achieve cost-effective and affordable energy and demand savings by continuing the 2016-2018 Plan approach, ensuring program stability, minimizing costs, and aligning with legislative requirements and ratepayers' best interests.

2018 Page 16 of 17 EFFICIENCYONE 2019 DSM PLAN FILING Appendix B 1 8. CONCLUSION 2 3 ENS’s 2019 DSM activities are based on a continuation of the approved 2016-2018 DSM 4 Resource Plan. The budget for 2019 is identical to the average annua...

AI summary The 2019 DSM Resource Plan by ENS is a continuation of the 2016-2018 plan, with a budget identical to the average annual budget of that period. The 2019 plan uses a combination of extending previous plans, updating cost-effectiveness insights, and adapting based on EfficiencyOne's experience, rather than conducting new resource modelling.

Attribution Unnamed: 0 Unnamed: 1 Unnamed: 2 Unnamed: 3 Unnamed: 4 Unnamed: 5 Unnamed: 6 Unnamed: 7 Unnamed: 8 Unnamed: 9 Unnamed: 10 Unnamed: 11 Unnamed: 12 Unnamed: 13 Unnamed: 14 Unnamed: 15 Unnamed: 16 Unnamed: 17 Unnamed: 18 Unnamed:...

AI summary The table presents data related to rate class numbers and participant counts, including alternate scenarios, though many entries are missing or undefined. This information may be relevant to regulatory proceedings involving rate calculations and participant metrics.

Verification Review of Program Year 2017 Evaluation Results Report for the Nova Scotia Utilities and Review Board H. Gil Peach & Associates LLC H. Gil Peach, PhD C. Eric Bonnyman, MS Jai J. Mitchell, BS . May 2, 2017 Vision Statement To be...

AI summary This document is a verification review of the 2017 Program Year Evaluation Results for the Nova Scotia Utilities and Review Board, conducted by H. Gil Peach & Associates LLC. The report outlines the firm's mission, goals, and methodology for evaluating program performance.

0716 EIN: 11-3783390 Fax: (503) 946-3064 E-mail: [email protected] ii Verification Review Of Program Year 2017 Evaluation Results

AI summary The document outlines a verification review of Program Year 2017 evaluation results, focusing on assessing the accuracy and validity of program performance metrics and outcomes.

iii Table of Contents I. Introduction & Overview ............................................................................................1 A. Demand-Side Management as Continuous Improvement .....................................1 1. Pl...

AI summary The document outlines a regulatory proceeding's structure, focusing on demand-side management (DSM) planning, implementation, evaluation, and savings verification. It addresses net demand reduction, energy savings, market transformation, climate mitigation, and evaluation standards, including technical resources and transparency requirements.

......................................................................... 17 4. Technical Resource Manual ................................................................................ 17 5. Evaluation Guidelines ...........................

AI summary The document outlines a structured evaluation framework for energy efficiency programs, including technical resources, evaluation guidelines, process evaluation methods, savings verification approaches, general findings, recommendations, and detailed reviews of specific initiatives like appliance retirement, instant savings, home energy assessments, and residential efficiency programs.

................. 10 Figure 5: Rank-Ordered Lifetime Net Energy Savings. .................................................... 11 Tables Table 1: Net Demand Reduction at Generator. ..............................................................

AI summary This report, conducted by H. Gil Peach & Associates for the Nova Scotia Utility and Review Board, verifies electricity savings and demand reduction data from Efficiency Nova Scotia's Demand Side Management (DSM) programs. It focuses on evaluating Econoler's impact assessments and may recommend adjustments to the savings estimates.

es created by evaluation consultants engaged by Efficiency Nova Scotia.1 For this reason it is focused on impact evaluation. The report also provides limited comments on other parts of the evaluation. The examination is based on careful re...

AI summary This document outlines an evaluation of Efficiency Nova Scotia's Demand-Side Management (DSM) programs, focusing on impact assessment, methodology review, and quality assurance processes. It highlights the transition of DSM administration from Nova Scotia Power to Efficiency Nova Scotia and notes the program's maturity after a decade of implementation.

tfolio. 2 If you already know this, you might skip to the summary of evaluation results in the next section. 1 Page Verification Review of Program Year 2017 Evaluation Results goal continuous process improvement.3 Figure 1 shows the DSM cy...

AI summary The document outlines the Demand-Side Management (DSM) cycle, emphasizing continuous improvement through four focal areas. It highlights the Planning phase's role in developing program portfolios, incorporating new knowledge, and conducting benefit/cost tests to guide implementation. The 2017 evaluation results are reviewed as part of this process.

forward from other steps in the process. The Plan provides a high-level blueprint for Implementation in the form of a program portfolio, the results of benefit/cost tests and specific program plans. 2. Implementation The Implementation ste...

AI summary The document outlines the DSM cycle's Implementation and Evaluation phases. Implementation involves adapting programs based on field realities and feedback, while Evaluation provides annual assessments and ongoing corrections. The DSM Plan serves as a blueprint with program portfolios and benefit/cost analyses.

River Project; Take a Walk on the Applied Side,” Pp. 112-118, The Rural Sociologist, Vol.5, No. 2, May 1985. 3 Page Verification Review of Program Year 2017 Evaluation Results produces independent measurements, observations, site inspectio...

AI summary The document outlines the verification process for Program Year 2017 evaluation results, emphasizing independent data collection methods and savings verification. Efficiency Nova Scotia and Econoler evaluated demand reduction, reporting 23.679 MW from DSM programs and 4.028 MW from Codes & Standards, with a 47%/53% split between residential and BNI sectors.

n in Table 1, the DSM portfolio net energy savings at the generator (GWh) is split about 41% to the Residential sector and about 59% to the business, non-profit and institutional program sector (BNI). In Figure 2, the DSM program component...

AI summary The DSM portfolio's net energy savings are split 41% residential and 59% BNI. Key programs like Green Heat and Business Energy Rebates contribute significantly to peak demand reduction. Top programs account for 80% of first-year energy savings, with lifetime savings analyzed in Figure 4.

rcentages, the top approximately 40% of DSM portfolio programs produce 80% of net first-year energy savings. 7 Page Verification Review of Program Year 2017 Evaluation Results indicated in Figure 4, both as a multiple of first-year net ene...

AI summary The top 40% of DSM portfolio programs generate 80% of first-year net energy savings, per evaluation results. Table 2 quantifies 2017 net energy savings at the generator, with Figure 4 illustrating multiplier effects on savings. The text focuses on verifying program performance metrics.

Direct Small Business Energy Solutions 7.887 7.887 Installation Installation BNI Subtotal 76.877 Overall DSM Portfolio Total 130.749 Codes & Standards 21.914 Total (DSM Portfolio plus Codes & Standards 152.663 Table Notes Due to year to ye...

AI summary The 2017 DSM program evaluation highlights energy savings contributions from Business Energy Rebates, Custom, Instant Savings, and Residential Efficient Product Installation, accounting for 73% of total net lifetime GWh. The report notes no SEM results in 2017 due to timing, with data sourced from Table 5 of the 2017 DSM Program Evaluation Executive Summary.

Verification Review of Program Year 2017 Evaluation Results Net Evaluated Lifetime Energy Savings (at Generator)

AI summary The document presents a verification review of Program Year 2017 evaluation results, focusing on net evaluated lifetime energy savings measured at the generator level. The analysis assesses the effectiveness of energy efficiency programs in achieving long-term savings, though specific findings or conclusions are not detailed in the provided text.

Direct Small Business Energy Solutions 100.854 100.854 Installation Installation BNI Subtotal 966.078 Overall DSM Portfolio Total 1620.838 Table Notes Due to year to year timing, there were no SEM results in 2017. Source: Data from Table 5...

AI summary The text presents a table summarizing energy savings data from the 2017 DSM program evaluation, including a BNI subtotal and overall DSM portfolio total. It notes the absence of SEM results in 2017 due to timing issues and cites the 2017 DSM Program Evaluation Executive Summary as the source.

nerator (GWh). 10 P a g e Verification Review of Program Year 2017 Evaluation Results Table 3: Lifetime Multiple (%) and Lifetime Additional Net Energy Savings (GWh). Net Lifetime Lifetime DSM Component Annual Lifetime Multiple Additional...

AI summary The document presents a verification review of Nova Scotia's 2017 Demand Side Management (DSM) program evaluations, quantifying energy savings across initiatives like appliance retirement, home assessments, and business rebates. It highlights lifetime net energy savings metrics and efficiency rankings for each program component.

749 1620.838 1240% 1,490 Figure 5: Rank-Ordered Lifetime Net Energy Savings. 11 P a g e Verification Review of Program Year 2017 Evaluation Results This evaluation of net lifetime energy savings developed by Econoler is very useful in unde...

AI summary The document reviews Econoler's evaluation of the DSM program's 2017 results, emphasizing the importance of understanding measure lifetimes and persistence for accurate cost-effectiveness analysis and long-term planning. It highlights how net lifetime energy savings require rethinking valuation frameworks to account for long-term benefits versus upfront costs.

nt, while cost often occurs in the first year. Program performance is not a single year performance. For customer value and value in resource planning, DSM components are meant to engage the future. 10 Hoffman, Ian M., Steven R. Schiller,...

AI summary The text emphasizes that Demand Side Management (DSM) components must consider long-term impacts beyond initial costs, citing Lawrence Berkeley Laboratory (LBL) research. It outlines a two-part evaluation of market transformation programs, focusing on resource acquisition effects and planned initiatives, while referencing dialectical principles to explain paradigm shifts in energy savings.

berto Sanchez Rodriguez, Xuemei Bai, Aliyu Salisu Barau, Karen C. Seto and Shobhakar Dhakal, “Locking in Positive Climate Responses in Cities.” Nature Climate Change, March 2018, 8:3, Pp. 174-177., 14 P a g e Verification Review of Program...

AI summary The text emphasizes the importance of independent evaluation in Demand Side Management (DSM) programs to ensure integrity and avoid conflicts of interest. It highlights that regulatory requirements since 1992 mandate independent evaluations, citing the Econoler evaluation as an example meeting this standard.

rement.19 The Econoler evaluation is an independent evaluation that meets this standard. 18 A useful insight from using a climate adaptation perspective is that it highlights the risk in adoption of “low-hanging fruit” type program strateg...

AI summary The text discusses the Econoler evaluation's role in meeting evaluation standards, highlights risks of short-term 'low-hanging fruit' energy efficiency strategies under climate adaptation, and distinguishes between Automated M&V and Automated Evaluation. It references Lawrence Berkeley Laboratory's research on automating M&V for whole-building approaches.

, 2014-2016 Assessment of Automated M&V Methods, Summary of Work (http://eis.lbl.gov/auto- mv.html). 23 Some vendors will claim that they have proprietary and confidential methods based on engineering models perhaps combined with big data...

AI summary The text critiques proprietary methods in automated M&V (Monitoring and Verification), emphasizing transparency and validation. It warns against over-reliance on algorithmic approaches in 'Evaluation 2.0,' which may compromise accuracy and invite misstatements of energy savings. Physical measurements and evaluator independence are stressed as critical, with the 2017 Econoler evaluation cited as a transparent example.

of the evaluator, including the evaluator’s independence in control of methods of analysis for evaluation. The 2017 Econoler evaluation has no problems of this kind. 3. Transparency DSM evaluation plans and reports are made public and are...

AI summary The evaluation of the 2017 DSM program emphasizes the independence of evaluators, transparency in reporting, and the use of a Technical Resource Manual (eTRM). Efficiency Nova Scotia’s eTRM is noted as a technical advance, while Econoler’s independent reviews are highlighted to prevent circularity in energy savings calculations.

re values (such as unitary savings and effective useful life). It will be important to continue this independent check to avoid circularity based on the eTRM. 5. Evaluation Guidelines Many jurisdictions also have a set of evaluation guidel...

AI summary The text discusses evaluation guidelines for DSM programs, noting the shift from older guidelines to the Uniform Methods Project standards. It highlights weaknesses in process evaluation efforts compared to market and impact evaluations, recommending increased focus on process evaluation for future assessments.

ion Review of Program Year 2017 Evaluation Results Table 4: Evaluation Effort for 2016 & 2017. II. Savings Verification Approach The savings verification review was conducted as follows: • During the year, we conducted a set of due diligen...

AI summary The document outlines the 2017 program year evaluation process, emphasizing savings verification through site visits, tracking system reviews, and eTRM material analysis. Focus areas include assessing installed energy savings, demand reductions, and system performance, with stakeholder communication highlighted.

t are the savings that should be realized when each of the projects operates for a full year.] These numbers come from the tracking system and from the evaluator’s adjustments. • Generally, we did not check the evaluator’s mathematical cal...

AI summary The evaluation of Efficiency Nova Scotia programs in 2017 is deemed comprehensive and methodologically sound, with Econoler's analyses adhering to industry frameworks. Key findings highlight the value of lifetime net energy savings assessments at the generator level, enhancing the understanding of program impacts. The review focused on methodological rigor rather than mathematical recalculations, except in limited cases.

g 20 P a g e Verification Review of Program Year 2017 Evaluation Results energy savings for 2017. This lifetime analysis is useful in understand the full value of the programs. SVF-3: Market transformation for 2017 is well addressed. Marke...

AI summary The document reviews the 2017 evaluation of energy programs, confirming Econoler's independent assessment of market transformation, climate mitigation, and transparency. It notes the evaluation's reliance on direct analysis rather than automated tools and its appropriate use of eTRM data.

tically rely on the eTRM, though sometimes values from the eTRM may be used. The evaluator independently is responsible for unitary savings values and effective measure lives. The use of a Technical Resource Manual (here an electronic TRM...

AI summary The text emphasizes the importance of independent evaluation to avoid circularity when using the eTRM for savings calculations. Econoler is credited with independently verifying unitary savings values and effective measure lives, ensuring evaluation integrity. The Savings Verification study recommends accepting 2017 evaluation results for all programs.

the 2017 evaluation results for energy savings and for demand-reduction for all programs. 22 P a g e Verification Review of Program Year 2017 Evaluation Results 2. Savings Verification Recommendation No. 2: Process Evaluations. Process eva...

AI summary The 2017 evaluation of energy savings and demand-reduction programs highlights recommendations for improving process evaluations and prioritizing non-lighting energy efficiency measures. The Savings Verification Review (SVR) urges increased process evaluation efforts and stronger promotion of non-lighting initiatives in future DSM cycles.

measures for the next DSM cycle . 23 P a g e Verification Review of Program Year 2017 Evaluation Results V. Individual Program Component or Initiative Review Econoler conducted evaluations of ten DSM components across the residential and B...

AI summary The document outlines the evaluation of ten Demand Side Management (DSM) program components in 2017, focusing on impact assessment. Key programs include the Appliance Retirement Program, Home Energy Assessment, and BNI initiatives. The review includes a pilot project involving the Halifax Regional School Board (HRSB) and evaluates savings from appliance retirements and replacements.

lot project, appliances owned by the Halifax Regional School Board (HRSB) were audited and collected by the Appliance Retirement Program. Savings from these two additions are included in this program. The 2017 evaluation used documentation...

AI summary The 2017 evaluation of the Appliance Retirement Program included audits of appliances owned by the HRSB, visits to recycling facilities, and improvements to metering protocols. Net energy and peak demand savings were found to be higher than previously reported, with recommendations for continued improvements in metering and tracking processes.

aluator also recommends improvement to the Efficiency Nova Scotia tracking process by clearly identifying appliances retired through the HRSB pilot. The logic of these recommendations is self-evident. Although reduced in comparison with th...

AI summary The evaluator recommends improving Efficiency Nova Scotia's tracking process by clearly identifying retired appliances through the HRSB pilot. The 2017 evaluation of Instant Savings shows slightly reduced sales and energy savings due to a price floor for LEDs, which caused some retailer frustration. The evaluation is considered strong and continues an improved measurement approach.

ive values are fully within the scope of an independent evaluator. The analytic detail and coverage of the evaluation of this program is complete within this context. This is an excellent evaluation. 3. Home Energy Assessment Home Energy A...

AI summary The Home Energy Assessment (HEA) program provides rebates and zero-interest financing to homeowners for energy efficiency measures, including insulation, heating systems, and water heating. The evaluation of the program's 2017 results includes performance measurement through 'test-in/test-out' residential audits and blower door testing conducted by energy advisors.

p://energy.gov/sites/prod/files/2015/02/f19/UMPChapter21-residential-lighting-evaluation- protocol.pdf). 26 P a g e Verification Review of Program Year 2017 Evaluation Results For 2017, the evaluation includes a documentation review, inter...

AI summary The evaluation of the 2017 program includes documentation reviews, interviews, surveys, and simulations. It highlights the need to simplify the incentive structure and conduct a new billing analysis due to changes in software. Health and safety training is also recommended, particularly regarding mold issues.

heated homes in this program in the future. The methods and calculations performed by the evaluator are appropriate to this program type and within the scope of an experienced independent evaluator. In this Savings Verification study, we a...

AI summary The document discusses concerns regarding the independence of post-installation inspections in the Green Heat program, highlighting past customer complaints and the need for on-site verification by an entity independent of the dealer or contractor. It emphasizes the importance of ensuring proper installation and equipment qualification.

th other program components, customer say they want to rely on the expertise of Efficiency Nova Scotia. 29 P a g e Verification Review of Program Year 2017 Evaluation Results SVR-5: In future evaluations, the evaluator should report on the...

AI summary The document discusses the Verification Review of Program Year 2017 Evaluation Results, emphasizing the need for improved inspection and verification processes for Green Heat installations. It also covers the Residential Efficient Product Installation program, including methodology used for evaluation, such as participant surveys, on-site visits, and metering studies to assess energy savings.

the accuracy of the energy and peak demand savings calculations for residential lighting measures for Nova Scotia. Investigation and estimation of spillover and interactive effects are also included. Econoler conducted a careful study of p...

AI summary This text discusses the evaluation of energy and peak demand savings from residential lighting measures in Nova Scotia, including challenges with smart power controllers, interactive effects of LEDs, and detailed analysis of product effectiveness and free-ridership. It emphasizes the complexity of measurement and evaluation processes involved.

high, relying on major engineering and evaluation organizations structured in a team approach. It takes several years to achieve command of the levels of complexity and interaction demonstrated here. The methods and calculations performed...

AI summary The document evaluates the New Residential Program in Nova Scotia, highlighting the use of energy-saving technologies and the need for documentation of energy savings relative to non-incandescent lamps. It also discusses the program's incentive structure and evaluation methods, including interviews and surveys.

Separately, 4 participants from the passive house pilot were interviewed. Free-ridership and spillover were 31 P a g e Verification Review of Program Year 2017 Evaluation Results analyzed. The savings analysis was based on a regression app...

AI summary The document discusses the evaluation of the 2017 DSM program, highlighting issues with low incentives and complex requirements. Econoler recommends a process evaluation, benchmarking study, and new billing analysis to improve the program. The evaluation methods are deemed appropriate and thorough.

ng, IT & datacenters, variable frequency drives, water heating, solar thermal, pumping and pool equipment. 32 P a g e Verification Review of Program Year 2017 Evaluation Results For 2017, this is a reduced evaluation in that there was an e...

AI summary The 2017 evaluation of the program involved a reduced assessment compared to 2016, with data verification and interviews conducted with participants and distributors. The study found that LEDs are approaching maturity in the BNI sector. Free-ridership and spillover effects were calculated using a combination of interviews, and effective measure life reviews were conducted.

d with market evolution for this program component, Econoler concludes that “…LEDs continue to be in a growth phase, but are approaching maturity as the BNI sector increasingly adopts these products.” Econoler recommends continued tracking...

AI summary Econoler evaluates the BNI Efficient Products Rebates program, noting LEDs are nearing maturity and recommending tracking market indicators, improving data collection for rebate participants, and addressing specific technologies. The BNI Custom Retrofit program offers technical and financial support for business, non-profit, and institutional customers.

For new construction, only 6 projects were completed during 2017. Econoler conducted a detailed technical review for 5 of these. Work included correction of simulation models used by the consultants. For building optimization, only 6 proje...

AI summary In 2017, only six new construction projects and six building optimization projects were completed under the BNI Custom program. Econoler conducted technical reviews and found significant errors in energy savings modeling, reducing net energy savings by 8% and net peak demand reduction by 38%. Recommendations for the next DSM cycle include continuing Efficiency Nova Scotia's efforts.

continue 32 Strategic Energy Management (SEM) is not included since SEM did not report savings in 2017. 34 P a g e Verification Review of Program Year 2017 Evaluation Results

AI summary The document discusses the verification review of Program Year 2017 evaluation results, noting that Strategic Energy Management (SEM) is not included as it did not report savings in 2017.

ticipation in new construction, development of a change tracking mechanism for retrofits, specifying and tracking retrofit effective useful life in the Custom tracking report, and addition of adjustment assumptions to modify model outputs...

AI summary The text discusses recommendations for improving participation in new construction and retrofit projects, including a change tracking mechanism and adjustments to model outputs. It also evaluates the BNI Custom program and the EMIS sub-program, highlighting the role of EMIS in providing real-time energy data and the findings from the 2017 evaluation.

2017 two participants follow an alternative Affordable Multifamily Housing pilot track. This track works like SBES but basis decisions on the affordable housing criterion rather than on energy usage. Econoler reviewed Efficiency Nova Scoti...

AI summary Econoler conducted a desk review and on-site assessments to evaluate Efficiency Nova Scotia's 2017 DSM program results. The review identified the need for improved procedures in monitoring complex custom projects and non-lighting measures, as well as better data collection practices to ensure accurate savings calculations.

street-lighting • New building energy code for houses and small building • New building energy code for large buildings, condos and apartments. • Clothes washers – Tier 1 • Refrigerators. For 2017, the evaluation approach was secondary dat...

AI summary The document outlines the evaluation of Efficiency Nova Scotia's 2017 program results, emphasizing the use of secondary data analysis, interviews, and market size assessments. It highlights the evaluation's alignment with industry standards and its focus on energy savings, climate mitigation, and market transformation.

well addressed. (Page 21) SVF-4: The evaluation appropriately addressed climate mitigation for 2017. (P. 21) SVF-5: The 2017 evaluations conducted by Econoler are appropriately independent. (P. 21) SVF-6: The 2017 evaluation does not rely...

AI summary The document reviews the 2017 energy savings evaluations conducted by Econoler, confirming their independence, transparency, and adherence to evaluation guidelines. It also outlines recommendations from the Savings Verification study, including acceptance of evaluation results, increased process evaluation efforts, and improved marketing of non-lighting measures.

REDACTED (CONFIDENTIAL INFORMATION REMOVED) 2019 DSM Plan Application, 2017 Annual Progress Report and 2017 Evaluation Reports (NSUARB M08604) NSPI Responses to SBA Information Requests NON-CONFIDENTIAL

AI summary The document references the 2019 DSM Plan Application, 2017 Annual Progress Report, and 2017 Evaluation Reports (NSUARB M08604), along with NSPI's responses to SBA Information Requests, indicating regulatory oversight of demand-side management programs and compliance with information disclosure requirements.

proposal to annually track comparisons of DSM cost recoveries to DSM 27 expenditures, and then adjusting any required variances during the next 28 GRA.2 1 Nova Scotia Power Inc. – DSM Cost Allocation and Recovery – M07151/E-R-15, December...

AI summary The text proposes annually tracking DSM cost recoveries against expenditures and adjusting variances in subsequent GRA.2 reviews. It references prior regulatory matters (M07151, NSUARB M08604) and documents including the 2019 DSM Plan Application and 2017 Evaluation Reports, with NSPI responding to SBA information requests.

E1 (SBA) IR-02 Page 1 of 1 EfficiencyOne – 2019 Demand Side Management (DSM) Resource Plan M08604 (E-ENS-R-18) E1 Responses to Small Business Advocate NON-CONFIDENTIAL 1 Request IR-03: 2 3 Please explain the method used by EfficiencyOne in...

AI summary EfficiencyOne explains its method for allocating the 2019 DSM Plan's budget and targets by referencing the UARB-approved 2016-2018 DSM Plan, 2014-2017 results, program participation data, and evaluation outcomes. A spreadsheet was used to compare historical data and draft 2019 budgets reflecting anticipated program uptake and diversification goals.

tudies in 2011 and 2016 related 19 to estimating the market penetration of lighting measures. Please refer to Attachments 20 1 and 2 of this IR response for these studies. 21 22 As part of the 2017 Evaluation of programs, the Evaluation Co...

AI summary The response references studies on LED market penetration from 2011 and 2016, as well as 2017 evaluations analyzing Instant Savings and Business Energy Rebates programs. These evaluations assess market transformation and saturation of LEDs, with free-ridership levels discussed in part c). The document pertains to EfficiencyOne's 2019 DSM Resource Plan and responses to the Small Business Advocate.

E1 (SBA) IR-08 Page 4 of 5 EfficiencyOne – 2019 Demand Side Management (DSM) Resource Plan M08604 (E-ENS-R-18) E1 Responses to Small Business Advocate NON-CONFIDENTIAL Footnotes for Table 1 A dash (-) indicates no lighting measures are off...

AI summary Footnotes explain free-ridership level assumptions in EfficiencyOne's DSM Resource Plan, referencing annual DSM Evaluation Reports filed with the Nova Scotia Utility and Review Board. Notes clarify data sources, exclusions (e.g., low-income participants), and program merges (e.g., 2017 consolidation of Rental Properties & Condos into EPI).

888‐414 4‐1336 Date Filed: May 30, 2018 SBA IR-08 Attachment 1, Page 2 of 24 Table of Contents Page Introduction ............................................................................................................................ 2...

AI summary The document outlines the 2011 Socket Study, analyzing light bulb usage and energy efficiency initiatives. It includes sections on bulb counts, adoption intentions, and in-home verification processes, aiming to assess program effectiveness and customer engagement in energy efficiency measures.

NON-CONFIDENTIAL 1 Request IR-11: 2 3 Regarding the measures included in Direct Installation program 4 a) Please supply a table that contains list of measures offered via Direct Installation 5 program for every year during 2016 and 2017. I...

AI summary The response to Request IR-11 explains that the Direct Installation program's data is available at the end-use level rather than the measure level for 2016 and 2017. For 2018 and 2019, EfficiencyOne provides targets at the program and component level, not measure level, and refers to technical tables in the DSM Resource Plan for details.

organizations indicate that they would participate in the future, coupled with the fact that nearly all Non-Participants hold a favourable overall opinion of Efficiency Nova Scotia. Among Participants, it is widely agreed that energy savin...

AI summary The text discusses the participation of small businesses in Efficiency Nova Scotia programs, noting high satisfaction with energy savings and bill reductions. Nearly all participants would recommend the programs and expect future participation, with positive overall opinions of Efficiency Nova Scotia.

mes resulting from program participation, six in ten organizations report having closely monitored energy consumption after taking part in an Efficiency Nova Scotia business program. Nearly all Participant organizations would recommend Eff...

AI summary The text highlights that most organizations participating in Efficiency Nova Scotia business programs would recommend them, with a variety of suggestions for improvement. However, promoting energy efficiency in communications and marketing is uncommon among participants, and organizations less inclined to participate or recommend programs are also less likely to promote energy efficiency.

ciency Nova Scotia? Q.28: [AMONG THOSE FAMILIAR WITH EFFICIENCY NOVA SCOTIA'S PROGRAMS (‘YES’ IN Q.10] Non-Participants: What is your overall opinion of Efficiency Nova Scotia?

AI summary The text asks individuals who are not participants in Efficiency Nova Scotia's programs for their overall opinion of the organization, indicating a focus on stakeholder feedback and program evaluation.

2017 SBES Study - Participants TABLE 16: TOTAL MENTIONS What was the main benefit, if any, to your company from participating in an Efficiency Nova Scotia business program? Probe: Any others?

AI summary The text references a 2017 SBES study and asks participants about the main benefit of participating in an Efficiency Nova Scotia business program, indicating a focus on program evaluation and stakeholder engagement.

2 10 8 14 6 9 9 8 7 12 Less than expected 7 7 7 9 5 0 13 4 8 5 0 9 6 12 6 9 0 11 6 4 17 4 12 About as expected 83 81 84 84 81 84 78 86 80 90 70 86 82 86 82 84 86 81 84 87 75 88 76 There were no expectations 1 0 2 2 0 0 3 0 2 0 0 1 2 0 2 0...

AI summary The table presents survey results on customer satisfaction with Efficiency Nova Scotia business programs, showing that most respondents indicated they would 'about as expected' recommend the programs, with a few indicating they would 'less than expected' or 'probably not' recommend them.

needs of the market by researching and launching new products, modifying incentive levels based on product prices, and monitoring codes and standards. 3.2.3 Barriers to Participation As part of its evaluation process, ENS’s external evalua...

AI summary The document discusses Efficiency Nova Scotia's (ENS) efforts to address barriers to participation in the Business Energy Rebates (BER) program through participant feedback and mitigation strategies, as outlined in Table 3.

405.9 GWh 62.4 MW 90% 80% 80% 23.7 MW 70% 66% Percent of 2016-2018 Target 130.7 GWh 60% 50% 40% 26.0 MW 30% 136.9 GWh 20% 10% 0% Energy savings (2016-2017 Energy savings (2016-2018 Peak demand savings (2016- Peak demand savings (2016- actu...

AI summary The text presents energy savings and peak demand savings data for 2016-2018, highlighting progress toward targets. It mentions that 2016 savings are verified, while 2017 savings are evaluated and pending verification. The data is presented in percentages and gigawatt-hours.

hich did not receive an impact or market 6 evaluation since it did not claim any savings in 2017 (please refer to section 1.3.2 7 for further discussion). 8 9 Impact evaluations assess net incremental energy and net peak demand savings, wh...

AI summary The document discusses the evaluation of program components, including impact and market evaluations, which assess energy savings and market conditions. It also mentions process evaluations for specific programs and support processes such as marketing, training, and incentive setting.

o Business development; and 26 o Incentive setting (in accordance with ENS’s Incentive Setting 27 Implementation Plan). 5 This is as a result of consistent participant survey results in previous years due to a stable market and the maturit...

AI summary The document discusses Efficiency Nova Scotia's 2017 DSM annual progress report, including program evaluations, rate-class expenditures, and the rationale for reduced impact evaluations in certain program components such as Appliance Retirement and Instant Savings.

38,101 2 Includes estimates of low-income participation in program components where income is not disclosed. Estimates reported in previous quarters may have been updated and are 3 reflected in the YTD totals. Energy and demand savings are...

AI summary The text provides details on low-income participation estimates in energy efficiency programs, noting assumptions made when income is not disclosed. It highlights that participation rates are based on assumptions and may be revised in the future as methodologies improve. Savings and expenditures are calculated with considerations for free-ridership and spillover effects.

7 In 2017, Regulatory Affairs activities included: 8 • engaging with regulatory stakeholders, the DSM Advisory Group and consultants, 9 as appropriate, on topics including: 10 o the development approach for the 2019 DSM Resource Plan; 11 o...

AI summary In 2017, Regulatory Affairs engaged with stakeholders on topics such as the 2019 DSM Resource Plan, non-energy benefits quantification, and the Incentive Setting Methodology Study. Activities included UARB reporting, stakeholder meetings, audits, and data provision from NS Power.

. 30 ENS will provide its mid-course adjustments for 8 2018 by program and their customer rate class impacts within its 2018 Q1 DSM Report, 9 due to be filed in May, 2018. 10 11 In 2018, ENS will work with its Evaluation Consultant to desi...

AI summary ENS plans to provide mid-course adjustments for 2018 by program and their customer rate class impacts in its Q1 DSM Report, due in May 2018. ENS will continue working with its Evaluation Consultant to design evaluations, conduct condensed evaluations for mature programs, and implement lean process improvements to enhance efficiency and participant experience.

ENS will also work with its Evaluation Consultant again in 2018 to establish an 21 appropriate scope for the Codes and Standards evaluation. 22 23 Regulatory Affairs 24 25 Regulatory Affairs activities will include reporting to the UARB an...

AI summary Efficiency Nova Scotia (ENS) will collaborate with its Evaluation Consultant in 2018 to define the scope of the Codes and Standards evaluation. Regulatory Affairs activities include reporting to the UARB and stakeholders on various topics, including DSM Resource Plans, financial reports, and non-energy benefits quantification.

• quantification of non-energy benefits for cost-effectiveness testing purposes; 3 and 4 • addressing evaluation and verification recommendations. 5 6 In the first quarter of 2018, ENS consulted with DSM Advisory Group stakeholders to 7 de...

AI summary The document discusses Efficiency Nova Scotia's (ENS) work on the 2019 DSM Plan, in line with the Electricity Efficiency and Conservation Restructuring (2014) Act. It also mentions planned and actual DSM expenditures for 2015-2017 and highlights ENS's collaboration with the DSM Advisory Group and Evaluation and Verification Consultants.

Result’s 17 recommendations. The incentive setting and review process uses a Consolidated 18 Calculator, which was developed by ENS further to recommendations from CLEAResult. 19 20 ENS also performed a financial simulation for all measure...

AI summary The document outlines Efficiency Nova Scotia's (ENS) implementation of incentive setting and review processes, including the use of a Consolidated Calculator and a financial simulation conducted in 2017 based on CLEAResult recommendations. An evaluation of ENS's progress was also conducted and filed as part of the 2017 Evaluation Report.

Filed: May 30, 2018 SBA IR-17 Attachment 1, Page 73 of 92 Table 1 Attachment 1, Page 1 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary The document provides an update on the implementation of recommendations from a 2013-2015 evaluation, likely related to energy efficiency or regulatory processes in Nova Scotia. It appears to be part of a submission or filing in a regulatory proceeding.

includes technical specifications of equipment, answers to pre-screening and will integrate others in 2018. questions and savings equations. 2013 Evaluation Evaluate the benefits/costs in acquiring a Customer Relations Management TPE-R3 In...

AI summary The text discusses the integration of a Customer Relations Management (CRM) tool into the Dynamic DSM system by ENS in 2017 and 2018, as well as the adaptation of the DSMDS tracking system to a QA program implemented by ENSC in 2013. ENS agrees with recommendations to improve data collection and evidence-based decision-making.

high delta EnerGuide. The Evaluator commends ENSC on this action and therefore recommends that the results of these reviews be tracked in the DSMDS. 2013 Evaluation Adapt the next tracking system (DSMDS) to the QA program: A QA program PP-...

AI summary The Evaluator commends ENSC for adapting the DSMDS tracking system to the QA program and recommends tracking results within the system. ENS agrees and plans to implement the QA program fully by 2014, with interim tracking on a spreadsheet until full integration.

Attachment 1, Page 2 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, highlighting progress and ongoing efforts to address identified issues.

Attachment 1, Page 3 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, likely related to energy efficiency or regulatory processes in Nova Scotia.

Expected Evaluation/ Year Recommendation Text Source Status Comments Period of Verification Completion 2014 Evaluation Gather more data to conduct a conclusive calibration billing analysis in 2015. Custom NC-R1 Deferred ENS appreciates thi...

AI summary In 2014, an evaluation recommended gathering more data to conduct a conclusive calibration billing analysis in 2015. Despite 23 projects completed in 2012 and 2013, only seven had sufficient energy billing data for analysis. The inconclusive results led to the recommendation to continue the evaluation in future years and use Portfolio Manager for storing utility bill data.

using the tool. This will assist ENS in storing utility bill data until enough evaluation activity be continued in future evaluation. Other measures could be information is collected to conduct a future calibration billing analysis. taken...

AI summary The text discusses measures to improve the quality of evaluation results, such as requiring final energy simulations, enhanced building commissioning, and full energy simulation reviews to ensure accurate model inputs for future calibration billing analysis.

building commissioning a mandatory element of Custom NC and/or including a full energy simulation review to ensure the accuracy of the model inputs. 2014 Evaluation Adapt the next tracking system (DSMDS) to the QA program: A QA program PP-...

AI summary The document discusses the need to adapt the DSMDS tracking system to a QA program, with ENSC agreeing to implement this in 2014. The Evaluator recommends tracking QA results in the DSMDS and using a temporary spreadsheet until full integration is achieved. CEAs involved in assessments should be identified to support training.

Attachment 1, Page 4 of 20 Update on Implementation of 2013-2015 Evaluation Recommendations

AI summary This document provides an update on the implementation of recommendations from the 2013-2015 evaluation, likely related to energy efficiency or regulatory processes in Nova Scotia. It outlines progress made in addressing previously identified issues.

savings. The Evaluator also noticed, in a few cases, discrepancies between the evaluation. ENS also completed additional training of staff to ensure the calculation algorithms, the variable descriptions and the savings calculated for savin...

AI summary The Evaluator identified discrepancies in calculation algorithms and variable descriptions for savings calculations in the BER Instant Rebates program. ENS addressed these issues by training staff and updating evaluation categories to align with those used in the 2015 evaluation.

2013 Verification Research Questions. The next process evaluation should begin not at the level of official Process Evaluation Complete ENS agrees with this recommendation. Its Evaluator Complete documents, the expectations of executive ma...

AI summary The text discusses a process evaluation conducted by ENS, which agrees with a recommendation to start evaluations from a practical and empirical perspective. Research questions were developed to assess how well ENS's processes function in practice, considering all contractors and trade allies. ENS plans to continue this approach in future evaluations.

s responsibility for service relationships with the people and institutions that ENSC serves? What are the restrictions on scope and to what extent are they needed or unnecessary? How and how well does ENSC maintain good faith and quality...

AI summary The text raises questions about ENSC's service quality, organizational culture, and employee experiences, including concerns about good faith service, unintended consequences, work environment, career opportunities, and the presence of cynicism or irony within the organization.

tional processes or requirements. Are there experiences of unanticipated side- effects, and how have programs be adapted to encounter these realities?

AI summary The text raises questions about the unanticipated side effects of programs and how they have been adapted to address these issues, emphasizing the need for ongoing evaluation and adjustment in regulatory processes.

2013 Verification Boundary for Analysis. The next process evaluation should draw the boundary for the study Process Evaluation Complete ENS agrees with this recommendation and has Complete around ENSC plus its service delivery agencies and...

AI summary The process evaluation for ENSC and its service delivery agencies was completed, and ENS agrees with the recommendation to expand the boundary for future evaluations. The Evaluator used interviews and best practices literature, and ENS will ensure future evaluations include external partners.

Table 2 Attachment 1, Page 6 of 20 Update on Implementation of 2013-2015 Verification Recommendations Expected Evaluation/ Year Recommendation Text Source Status Comments Period of Verification Completion 2013 Verification Inclusion. Inter...

AI summary The text discusses the implementation of 2013-2015 verification recommendations, specifically focusing on the inclusion of all staff levels and delivery agents in interviews and surveys. ENS has implemented this recommendation in the Marketing evaluation and Practices & Procedures evaluation.

methodology involved interviews with ENS staff at all levels of management and interviews with partners and participants where appropriate. ENS will ensure all future process evaluations include interviews/surveys with all relevant groups...

AI summary The methodology involved interviews with ENS staff at all levels of management and with partners and participants where appropriate. ENS will ensure all future process evaluations include interviews and surveys with all relevant groups involved.

interviews/surveys with all relevant groups involved. 2013 Verification Confidentiality. Rather than focus groups, group settings or surveys which require personal Process Evaluation Complete ENS agrees with this recommendation. The Evalua...

AI summary The evaluation process conducted by the Evaluator involved individual interviews with ENS staff and others, ensuring confidentiality. ENS agrees with the recommendation to use dialog and confidentiality rather than focus groups or surveys requiring personal identification.

lose this information unless required by law. 2013 Verification Readability. The next organizational level process evaluation should be written in a style that Process Evaluation Complete ENS agrees with this recommendation and has Complet...

AI summary The document recommends that future process evaluations by ENS be written in a more readable style, understandable to Nova Scotia residents without special training. It suggests minimizing technical terms like balanced scorecards and focusing on storytelling in everyday language to better communicate the organization's value and operations.

Table 2 Attachment 1, Page 7 of 20 Update on Implementation of 2013-2015 Verification Recommendations Expected Evaluation/ Year Recommendation Text Source Status Comments Period of Verification Completion 2014 Verification Develop evaluati...

AI summary The document discusses the development of evaluation guidelines as part of the 2013-2015 Verification Recommendations. ENS agrees with the recommendation and is in the process of developing guidelines based on the New York evaluation guidelines and appendices to set evaluation standards for future evaluations. The expected completion period is 2018.

ndation approach and expectations for future evaluations. certain evaluation standards for future evaluations. No. 3 2014 Verification Savings Weighted Measure Lives. The time dimension should be operationalized in the Savings Complete ENS...

AI summary The text discusses the need for a more comprehensive evaluation approach for energy efficiency programs, emphasizing the use of lifetime energy savings rather than just first-year savings. ENS agrees with this and started reporting lifetime energy savings in 2017, using a multiple baseline approach for evaluation.

conduct a future calibration billing analysis. 2015 Verification Savings Weighted Measure Lives. The time and savings dimension should be operationalized SV-2 Complete ENS agrees with this recommendation and began Complete in the indicator...

AI summary The text discusses the need for a more comprehensive approach to evaluating energy savings by incorporating a weighted measure of lives over time, rather than relying solely on first-year savings. ENS agrees with this approach and began reporting lifetime energy savings in 2017, using a multiple baseline methodology.

Continue to perform metering activity and keep improving its quality and effectiveness. This year, metering ARet-R1 Complete ENS agrees with this recommendation and met all 2017 metering Complete activity was focused on the small appliance...

AI summary The text discusses metering activities related to appliance retirement programs, highlighting improvements in metering protocols, challenges with data collection, and recommendations for better data precision in 2017. The Evaluator suggests using raw current transformer data and revising eligibility criteria to ensure more accurate impact evaluations.

Continue monitoring key market indicators to adapt program component offerings when needed. Increasing Instant-R1 Complete ENS agrees with this recommendation and completed it in 2017. Interviews Complete adoption of LEDs by consumers and...

AI summary The document discusses the need to continue monitoring market indicators for LED adoption, noting reduced barriers and increased sales. ENS agrees with the recommendation and completed it in 2017, while also piloting non-lighting products in the Instant Savings program.

Increase program component advertising to households and collaborate with retailers. Evaluation results showed GH-R2 In Progress ENS agrees with this recommendation. In 2017, it increased its marketing 2018 fairly high free-ridership level...

AI summary The evaluation of the Green Heat program highlights high free-ridership levels for rebated products, particularly heat pumps. It recommends increasing advertising to households and collaborating with retailers to improve outreach and promote higher efficiency models. ENS has taken steps to enhance marketing efforts, including digital ads and in-store materials.

Table 3 Attachment 1, Page 11 of 20 Update on Implementation of 2016 Evaluation Recommendations Expected Recommendation Text Source Status Comments Period of Completion Perform the MSHP billing analysis again in 2017. The billing analysis...

AI summary The 2016 evaluation recommended re-performing the MSHP billing analysis in 2017 to ensure accuracy with a larger sample and account for changes made by the Green Heat management team. ENS agrees with the recommendation and conducted the analysis as part of the 2017 Green Heat evaluation.

t is recommended that additional visits should be conducted in early 2017 so as to determine with confidence whether or not the installation rates have decreased in comparison with the previous years. Start measuring spillover in 2017. Spi...

AI summary The document recommends measuring spillover effects in the RP&C program starting in 2017 due to observed changes in product upgrades and installations, such as LED lamps and mini-split heat pumps, which may be influenced by market evolution and participant behavior.

still likely to start installing some of these measures without receiving any incentive encouraging them to do so. Therefore, the Evaluator recommends starting to measure spillover for RP&C in 2017. Continue conducting project reviews to a...

AI summary The Evaluator recommends starting to measure spillover for the Residential Program & Customer (RP&C) in 2017. It also suggests continuing project reviews to assess the quality of simulation files and improving training for Energy Assessors (EAs) to ensure consistency and accuracy in data recording and simulation processes.

oping such M&V procedures for Retrofit projects. The same procedures should be applied to BER projects generating 300,000 kWh and more to limit the corrections needed to the tracked energy savings. Conduct deeper market analysis to adapt t...

AI summary The document discusses the need for consistent M&V procedures for Retrofit and BER projects, deeper market analysis for BER Instant Rebates, and the impact of market evolution on LED sales and free-ridership. The Evaluator recommends adapting rebate offers and using multiple data sources to assess market transformation.

Table 3 Attachment 1, Page 13 of 20 Update on Implementation of 2016 Evaluation Recommendations Expected Recommendation Text Source Status Comments Period of Completion Identify the root causes of New Construction’s low participation. Alth...

AI summary The document discusses the need to identify the root causes of low participation in New Construction programs under the 2016 Evaluation Recommendations. ENS acknowledges the recommendation and plans to relaunch the service in Q2 2017, monitor feedback, and adjust as necessary. However, due to limited participant interviews, the Evaluator cannot provide detailed conclusions about barriers to participation.

evaluation, the Evaluator cannot provide insightful conclusions about the barriers encountered by participants. participation remains low through 2018, ENS and its Evaluator will consider Interviewing a larger group of participants and, ev...

AI summary The evaluation of the New Construction program indicates low participation through 2018, suggesting the need for further research into barriers and participant engagement. ENS has adopted an improved methodology for calculating interactive effects in Retrofit lighting projects starting in 2017 and plans to continue using it in 2018.

port is available in the technical literature. As new research is conducted on the impact evaluation of programs similar to EMIS, this methodology might be refined to capture all incremental savings. Investigate ways of using energy manage...

AI summary The text discusses the potential of energy management information systems (EMIS) to generate greater energy savings and suggests ways to enhance their use. It notes that while some participants did not implement new measures in 2016, ENS has taken steps to increase incremental savings and is working with participants to achieve additional savings.

eline energy regression in 2017. To avoid this in the future, systematic reviews of facility operational and equipment changes should be implemented by ENS to ensure the accuracy of reported savings. Continue to support SEM and identify po...

AI summary The text discusses the need for systematic reviews by ENS to ensure accurate energy savings reporting and highlights the success of the SEM program, with new participants enrolling in 2018 and ongoing recruitment efforts.

s also believed that participation in both EMIS and SEM would increase participant internal energy management capabilities and commitment, thereby helping achieve deeper and more sustainable savings. Consider extending the SEM cycle to eng...

AI summary The text suggests extending the SEM cycle to increase participant engagement and achieve deeper energy savings. ENS notes that it lacks sufficient information to analyze the cost-effectiveness of this change and plans to consider it in a future year.

Table 3 Attachment 1, Page 15 of 20 Update on Implementation of 2016 Evaluation Recommendations Expected Recommendation Text Source Status Comments Period of Completion Track incremental savings rather than cumulative savings. For continui...

AI summary The document discusses the implementation of a 2016 recommendation to track incremental energy savings rather than cumulative savings for continuing participants in the SEM program. ENS agrees with the recommendation and has modified the methodology to track incremental first-year savings based on a baseline period.

savings were the result of the sum of actions taken since the beginning of their participation and not only the additional actions taken since 2015. This was considered standard industry practice. Since ENS established all of its program c...

AI summary The document discusses how energy savings are calculated as incremental savings, considering ongoing support from ENS. It also highlights improvements in the Event Log for SEM impact evaluations, with ENS agreeing to create a more detailed template.

lements impacting the baseline and the energy efficiency measures implemented, rather than accuracy of savings calculations. categorizing them all as operational changes. Identify energy efficiency measures installed under ENS instant reba...

AI summary The text discusses the need to accurately track energy efficiency measures installed under ENS instant rebate programs to avoid double-counting savings, and the application of baseline energy regression adjustments to continuing SEM participants to ensure consistent evaluation methods.

Improve the application forms and the internal review procedure to ensure the accuracy of savings calculations SBES-R4 Complete ENS agrees with this recommendation and developed new participant Complete for DIY projects. The Evaluator foun...

AI summary The document highlights issues with the accuracy of savings calculations in energy efficiency programs, noting that variables used in the algorithm often do not reflect actual conditions. It recommends improving application forms and internal review procedures, as well as ensuring that SBEAs are qualified to conduct audits at industrial facilities.

iled: May 30, 2018 SBA IR-17 Attachment 1, Page 89 of 92 Table 4 Attachment 1, Page 17 of 20 Update on Implementation of 2016 Verification Recommendations Expected Recommendation Text Source Status Comments Period of Completion The Savings...

AI summary The document discusses the acceptance of 2016 evaluation results for energy savings and demand reduction across all programs, with no action required for this recommendation.

In future years, ENS should require the Evaluator to include a copy of the operative evaluation plan Recommendation Complete ENS agrees with this recommendation. In the 2017 DSM Programs Complete within the overall Executive Summary, suppl...

AI summary ENS agrees with a recommendation to improve the evaluation process by including a copy of the operative evaluation plan in the Executive Summary and explaining the rationale for changes in evaluation components. The Evaluator provided detailed descriptions of evaluation approaches in 2016 and 2017.

logy section of the corresponding program component report. In future years, ENS should require the Evaluator to provide a summary table of obtained confidence Recommendation Complete In each evaluation report's Methodology section, the Ev...

AI summary The text discusses recommendations for improving evaluation reports by requiring detailed methodology sections, including confidence and precision levels, population and sample sizes, and explanations for variations in results. It also references example calculations provided in an appendix.

x II of the 2017 DSM Programs Evaluation Executive Summary. In future years, ENS should specify rules to bound the dropping of evaluation components and to Recommendation In Progress ENS agrees with this recommendation. ENS has now complet...

AI summary The document recommends that ENS establish rules to limit the reduction of program impact evaluations, ensuring the validity and precision of evaluation results. ENS agrees and plans to refine its approach and finalize guidelines by 2018.

that take into account lessons learned during the 2017 evaluation process. carefully watched and rules set up to protect the continuing validity and precision of evaluation results. ENS should increase on-site inspection for quality contro...

AI summary The document discusses the need for ENS to increase on-site inspections for quality control in programs transitioning from ENS control to market reliance, specifically for the Small Business Energy Solutions program. It also highlights the importance of considering long-term savings estimates in evaluations to improve program comparisons and foresight.

viding savings estimates based on expected lifetimes to support comparisons General-SV-1 in each of its 2017 evaluation reports. among programs and to improve foresight. For program types that permit, ENS and the Advisory Group should cons...

AI summary ENS and the Advisory Group are considering the potential benefits and costs of introducing selected persistence evaluations to document lifetimes and persistence of savings. ENS appreciates the consideration but does not recommend pursuing it at this time, citing the 2017 review of Effective Useful Lives (EULs) and lifetime savings.

nue monitoring key market indicators for LEDs moving forward. In the 2017 evaluation for the Green Heat high efficiency mini-split heat pump measure, ENS should Green Heat-SV-R1 Complete ENS appreciates this consideration and discussed it...

AI summary The document discusses the evaluation of the Green Heat high efficiency mini-split heat pump measure in 2017, considering whether it should be structured as a market transformation evaluation rather than a resource acquisition evaluation. ENS integrated additional analyses of market indicators for program components like Instant Savings and Green Heat.

in the Verification Consultant's Recommendation EMIS-SV- 2. One way to more clearly communicate the approach is to show the results for each step and each EMIS-SV-3 Complete ENS agrees with this recommendation. To help communicate results...

AI summary The Verification Consultant recommends including detailed tables in the 2017 Custom Incentives (EMIS) evaluation report to clearly communicate the methodology and results of energy savings projects. ENS agrees with this recommendation, and the Evaluator added summary tables of all reviewed EMIS projects to enhance transparency while protecting participant information.

in the report and protection of participant information. Add the following SEM related terms to the Definitions (p. iii); Baseline regression, Projected baseline, SEM-SV-R1 In Progress ENS agrees with this recommendation. SEM did not achie...

AI summary The document discusses issues with the Strategic Energy Management (SEM) program, specifically the lack of savings achieved in 2017 and 2018, and the need to update definitions related to baseline regression and projected baselines in the report. Efficiency Nova Scotia agrees with the recommendation to define these terms in the 2018 evaluation report.

ate Filed: May 30, 2018 SBA IR-17 Attachment 1, Page 92 of 92 Table 4 Attachment 1, Page 20 of 20 Update on Implementation of 2016 Verification Recommendations Expected Recommendation Text Source Status Comments Period of Completion We sup...

AI summary The document discusses the deferred implementation of a recommendation to extend engagement with SEM participants, citing a lack of information for a full cost-effectiveness analysis. ENS plans to consider this in a future year.

consider implementing this recommendation in a future year. future year One way to more clearly communicate the approach is to show the results for each step and each SEM-SV-R3 In Progress ENS agrees with this recommendation. SEM did not a...

AI summary The text discusses the need for clearer communication of energy savings methodology by suggesting a table format to present projected baseline, adjustments, adjusted baseline, actual energy usage, and incremental savings. ENS agrees with the recommendation but notes that SEM did not achieve savings in 2017-2018, leading to no evaluation report being written for 2017. The Evaluator plans to refine the presentation of SEM data in the 2018 report.

CM 2018 SEM evaluation report. programs. Presumably such a table would also make clear how the value of 0.463 “Adjustment Ratio for Energy Savings” was calculated. Edits/Corrections: On P. 55, Figure 11, should the 2015 participant bar be...

AI summary The text references a 2018 SEM evaluation report and discusses edits and corrections to figures and tables within the report, including the number of participants in 2015 and a correction to a bullet point on page 64.

NON-CONFIDENTIAL 1 Request IR-04: 2 3 Regarding the residential sector results, please explain the changes in program measure 4 distribution that resulted in a higher demand-to-energy ratio. 5 6 Response IR-04: 7 8 The demand-to-energy rat...

AI summary The response explains that the higher demand-to-energy ratio in the residential sector is due to updated energy savings methodologies and the Green Heat program's disproportionate demand savings from mini-split heat pumps. Program evaluations since 2017 show higher demand-to-energy ratios than initially planned, affecting 2017 target comparisons.

NON-CONFIDENTIAL 1 Request IR-08: 2 3 Regarding the BNI sector results, please explain the changes in program measure 4 distribution that resulted in a higher demand-to-energy ratio. 5 6 Response IR-08: 7 8 The demand-to-energy ratio is in...

AI summary The response explains that the higher demand-to-energy ratio in the BNI sector is due to updated methodologies in savings estimation and the 2017 mid-course target methodology. The Custom program's success in New Construction also contributed to exceeding demand savings targets.

E1 (NSUARB) IR-12 Page 1 of 1 EfficiencyOne – 2019 Demand Side Management (DSM) Resource Plan M08604 (E-ENS-R-18) E1 Responses to Nova Scotia Utility and Review Board NON-CONFIDENTIAL 1 Request IR-13: 2 3 Page 9 also states that 2019 evalu...

AI summary EfficiencyOne (E1) responds to a request regarding condensed evaluations in the 2019 DSM Resource Plan (M08604). E1 explains that evaluations will be streamlined for mature program components, with the 2018 Evaluation Plan to be shared with the DSM Advisory Group in fall 2018, informed by prior feedback.

ished in this year’s evaluation for appliances retired through the pilot. The Evaluator also recommends improving the tracking process by clearly identifying the appliances retired through the pilot.

AI summary The evaluation recommends improving the tracking process for appliances retired through the pilot program to ensure clear identification of retired appliances.

Date Continue monitoring market indicators and collect additional information to further 2017 Instant-R2 In Progress ENS agrees with this recommendation. ENS is collaborating with the 2018 characterize the LED market. Because the lighting...

AI summary The text discusses the need to monitor the LED lighting market and collect data on market indicators, LED prices, and customer knowledge. ENS agrees with the recommendation and plans to develop an evaluation plan, conduct surveys, and update market evolution data for 2018.

Recommendation Text Source Status Comments Completion Date Conduct a process evaluation for NHC in 2018. During the 2017 evaluation, participants 2017 NHC-R1 In Progress ENS agrees with this recommendation. ENS is collaborating with the 20...

AI summary The text recommends conducting a process evaluation for the New Home Construction (NHC) program in 2018, following criticisms from participants and builders regarding incentive levels and participation processes. The evaluation would include benchmarking and consultation with builders to optimize the program for market needs.

Continue to monitor the market evolution of LED lamps and adapt the BER offer when 2017 BER-R1 In Progress ENS agrees with this recommendation. ENS will continue to monitor market 2018 needed. The 2017 evaluation served to confirm solid ma...

AI summary The document discusses the ongoing monitoring of LED lamp market evolution and the need to adapt the BER offer. It highlights that market barriers are decreasing, and the BNI sector is increasingly adopting LED technologies. ENS will continue to evaluate and possibly adjust rebates based on market analysis and participant feedback.

times per year. Therefore, the Evaluator recommends that ENS identify BNI sectors that are currently underrepresented in the Mail-in service and that could potentially generate significant savings. Continue and improve the tracking of BER...

AI summary The Evaluator recommends that ENS identify underrepresented BNI sectors in the Mail-in service and improve tracking of BER Instant Rebates participants to better assess savings and refine impact analysis. ENS agrees and will add clauses to contracts to ensure data collection and work with the Evaluation Consultant to provide detailed participant information.

Ensure that the tracking sheet is complete and accurate. While reviewing the project 2017 EMIS-R1 In Progress ENS agrees with this recommendation. ENS will update its process for 2018 documentation, the Evaluator found several discrepancie...

AI summary The Evaluator found discrepancies between savings figures in the final tracking sheet and annual reports, and recommends updating the tracking sheet with final values from annual reports and tracking annual peak demand savings.

of the 2017 evaluation results for Recommendation 2017 No action required. energy savings and for demand-reduction for all programs. General-SVR-1 The Savings Verification study recommends increase in the level of effort in process Recomme...

AI summary The 2017 evaluation results for Recommendation 2017 indicate no action is required. However, the Savings Verification study recommends increasing the level of effort in the evaluation process for the next evaluation. ENS agrees with this recommendation and is developing a 2018 Evaluation Plan in collaboration with the Evaluation Consultant.

In future evaluations, the Evaluator should report on the structure and process of Recommendation 2017 In Progress ENS agrees with this recommendation. Inspection and/or verification of installed 2018 inspection/verification for Green Heat...

AI summary The document discusses the need for future evaluations of the Green Heat program, emphasizing the importance of inspecting and verifying installations. ENS agrees with the recommendation and is working with the Evaluation Consultant to define the scope of work for the 2018 Program Process Evaluation, including a review of quality assurance processes.

NON-CONFIDENTIAL 1 Request IR-03: 2 3 Page 14 rows 10 through 13 of the Evidence states, “In the context of the development of the 4 2019 DSM Resource Plan as a continuation plan, EfficiencyOne is proposing largely the same 5 performance t...

AI summary EfficiencyOne clarifies that the 2016-2018 DSM Resource Plan did not include total spending or ratepayer benefits as performance targets but as indicators. Savings from pilot programs (e.g., demand response) are included in 2019 targets, though lifetime GWh savings are not yet a target despite improved tracking capabilities.

NON-CONFIDENTIAL 1 Lifetime energy savings, total ratepayer benefits, total spending, and customer 2 satisfaction provide valuable context and insight into how ENS is performing, but 3 should be categorized as Performance Indicators as opp...

AI summary ENS argues that lifetime energy savings, ratepayer benefits, spending, and customer satisfaction should be Performance Indicators, not Targets, due to methodological adjustments, correlation with other metrics, budget constraints, and limited ex-post measurement feasibility. EfficiencyOne supports including lifetime savings in the next three-year plan but not as a Target yet.

does not reflect achievement of the DSM Plan 21 objectives in the same capacity as the two proposed Performance Targets. 22 23 b) No, these metrics were proposed and approved as Performance Indicators. 24 25 c) Confirmed – the value of 127...

AI summary EfficiencyOne clarifies that the DSM Plan's objectives are not met at the same level as proposed Performance Targets. Metrics were approved as Performance Indicators, with specific targets of 127.2 GWh for energy savings and 20.2 MW for demand savings. EfficiencyOne manages its portfolio to meet UARB-approved targets and adheres to the 2016-2018 Consensus Agreement requiring explanations for significant variances.

NON-CONFIDENTIAL 1 Request IR-07: 2 3 Please provide a summary of the current lighting saturation by measure and sector and how 4 this information informed the shifts EfficiencyOne implemented from lighting to non- 5 lighting measures in 2...

AI summary EfficiencyOne lacks detailed lighting saturation data by measure but used residential socket studies and 2017 evaluations to assess LED adoption. The 2017 report noted LEDs nearing maturity, prompting a shift to non-lighting measures in 2019. Lighting remains important for energy savings, but declining efficiency gains from LEDs influenced this strategic pivot.

0 Completing the Report 31 Accuracy of the Report 31 Close Out Performance Evaluations 31 11 LIST OF PROPOSAL SUBMISSION TABLES 32 Table 1: Company Information 32 Table 2: Acceptance of Terms and Conditions (Attachment 1) 32 Table 3: Proje...

AI summary This section outlines the structure and components of a proposal submission, including tables for company information, risk assessments, cost proposals, and attachments such as standard terms and conditions. It emphasizes the importance of accuracy and completeness in the report.

nd customer preferences in Nova Scotia. In addition, a modelling approach that can offer predictive functions with respect to market behaviour and measure uptake will be considered an asset. 4 SCOPE OF WORK EfficiencyOne is seeking an expe...

AI summary EfficiencyOne is seeking a DSM consultant to develop ENS’s 2020-2022 DSM Resource Plan, including a Preferred Plan and an Alternative Plan, both fully modelled. The consultant must use modelling software that allows for input and output inspection by EfficiencyOne, regulatory intervenors, and the UARB. The model must support Total Resource Cost testing for ENS’s six programs.

at the measure level. PORTFOLIO DESIGN CONSIDERATIONS Portfolio design should consider a balanced plan approach, which aims to achieve a balance among four general categories: 1. Savings: the plan should seek to maximize energy savings, in...

AI summary The document outlines portfolio design considerations for energy efficiency programs, emphasizing the need for balance among savings, costs and benefits, risk, and equity. It also discusses model access requirements and project phases, including the initial phase involving input data provision by EfficiencyOne.

returned surveys (up to the maximum). On a personnel-by-personnel basis, the Proponent(s) with the greatest number of returned surveys will receive full evaluation 21 Date Filed: May 30, 2018 Synapse IR-08 Attachment 1, Page 27 of 47 point...

AI summary The document outlines the evaluation process for proposals, emphasizing the importance of survey submission and the scoring methodology. It also explains that certain tables, such as Proposal Assumptions and Scope Review, will not be evaluated but will be reviewed after the selection process.

anager • Technical Lead • Software Lead 22 Date Filed: May 30, 2018 Synapse IR-08 Attachment 1, Page 28 of 47 The individuals that will be interviewed must be the same person that is identified in the Proponents’ Proposal. No substitutes,...

AI summary The document outlines the interview process for proponents, requiring the same individual identified in their proposal to attend in person. Interviews will last 20-30 minutes and focus on project experience, risk management, and project planning. Proponents failing to attend will receive a zero score, impacting their competitiveness.

32 Date Filed: May 30, 2018 Synapse IR-08 Attachment 1, Page 38 of 47 APPENDIX A – MODEL INPUT EXAMPLES AND DATA SOURCES EfficiencyOne can prepare input data in a user-friendly format and provide to the successful proponent. However, for c...

AI summary This document provides an example of input data format and sources for efficiency programs, referencing historical energy savings and program evaluations from the Nova Scotia Utility and Review Board (NSUARB). It also lists matter numbers for evaluation reports from 2015 to 2017.

operated under Enabling Strategies, as 28 opposed to within a program component; as such, their budgets are shown under the 29 Development and Research line item in Table 1, below. Date Filed: May 30, 2018 E1 (Synapse) IR-10 Page 1 of 9 Ef...

AI summary EfficiencyOne's 2019 DSM Resource Plan followed a continuation approach, and cost-effectiveness tests and lifetime benefits for program components were not completed. References are made to prior responses for detailed results.

E1 (Synapse) IR-10 Page 3 of 9 EfficiencyOne – 2019 Demand Side Management (DSM) Resource Plan M08604 (E-ENS-R-18) E1 Responses to Synapse Energy Economics NON-CONFIDENTIAL 41 b) Please refer to Tables 2 and 3, below, for the 2016 and 2017...

AI summary EfficiencyOne provides responses to Synapse Energy Economics regarding the 2019 Demand Side Management (DSM) Resource Plan, noting that cost-effectiveness tests and Lifetime Benefits are not calculated after a program year has concluded and are therefore not included in the tables provided.

NON-CONFIDENTIAL 45 Table 2: 2016 Results - by Program Component Incremental Incremental Lifetime Net Annual Net Incremental Annual Net Energy Demand First Year Lifetime Avoided Lifetime Investment Energy Savings at Eligible Participation...

AI summary The text presents a table summarizing the 2016 results by program component, including investment amounts, energy savings, and participation metrics. The table includes columns such as incremental annual net energy savings, unit costs, and avoided emissions, indicating a focus on energy efficiency and program evaluation.

NON-CONFIDENTIAL 52 Table 3: 2017 Results - by Program Component Incremental Incremental Lifetime Annual Net Incremental Annual Net Net Energy Demand First Year Lifetime Avoided Lifetime Investment Energy Savings Savings at Eligible Partic...

AI summary The text presents a table titled '2017 Results - by Program Component' that includes metrics such as investment, energy savings, and participation data for various programs. It outlines incremental annual net energy savings, unit costs, and other relevant financial and environmental indicators.

multifamily projects, it is not surprising that it is taking more than 2 years to figure out how to design and implement the Affordable Multifamily Housing program in the most effective way possible. Many of the projects that have been imp...

AI summary The Affordable Multifamily Housing program faces challenges in implementation due to small project sizes, lack of landlord incentives in rental properties, and insufficient rebates. Similar to the HomeWarming program, higher incentives are needed for success. Continuing the program in 2019 with adjustments is recommended over waiting for the 2018 evaluation.

on depends on the 2018 evaluation. Even if the evaluation leads to a time delay in the evolution of the Pilot, this will be a serious problem, especially given that the 2019 Plan is only for one year. We urge EfficiencyOne to commit to ano...

AI summary The Affordable Energy Coalition urges EfficiencyOne to extend a pilot program for another year to maintain momentum and ensure proper evaluation, emphasizing its benefits for low-income renters. They request the Board's support for this adjustment within the 2019 Plan.

ovided by DSM resources. Both of these points imply a significant level of 21 headroom available for procurement of additional DSM resources in the next 22 DSM Resource Plan. 23 Critically, E1 indicates that during 2018 and continuing into...

AI summary The text discusses headroom for additional DSM resources, negotiations for the 2020-2022 DSM Plan, and four conclusions/recommendations: prioritizing capacity savings, balancing lighting measures, improving transparency in DSM decision-making, and aligning research with resource planning.

~ Blackburn Law Jw1e 13, 2018 VIA EMAIL Ms. Doreen Friis Regulatory Affairs Officer/Clerk Nova Scotia Utility and Review Board 1601 Lower Water Street, 3rd Floor Halifax NS B3J 3 S3 Dear Ms. Friis: Re: M08604 - E-ENS-R-18 - EfficiencyOne -...

AI summary Blackburn Law submits concerns on behalf of the Small Business Advocate (SBA) regarding EfficiencyOne's 2019 DSM plan continuation, targeting $34.05M investment for 127.2 GWh energy savings. The SBA acknowledges the plan's continuation but highlights concerns requiring consideration during implementation and for the 2020-2021 DSM plan.

es to Syn.:pse Energy Economics, IR-OS(a}. 6 EfficiencyOne - 2019 Demand Side Management (DSM} Resource Plan M08604 (E-ENS-R-18) El Responses to Small Business Advocate, El(SBA) IR-07 (c} 7 Ibid. 21Page While the SBA acknowledges that the...

AI summary The SBA acknowledges the 2019 DSM plan's continuity but urges shifting focus from lighting to other areas. It requests separate reporting for SBES and Affordable Multifamily Pilot programs and improved documentation during applications. El provided studies aiding evaluation but needs to disclose such documents upfront in future submissions.

019 and reference was made to “potential” demand reduction pilot activities and “anticipated” operation of demand reduction pilots in the residential and business non-profit and institutional sectors. The Consumer Advocate notes there is a...

AI summary The Consumer Advocate highlights the opportunity cost of reallocating $1 million from existing programs to demand reduction pilots, citing reduced energy and demand savings. E1 revised its 2019 portfolio table to reflect these impacts. The text also emphasizes the need for a realistic rate and bill impact analysis (RBIA) to assess DSM program effects on customers and inform investment decisions.

to which EfficiencyOne has operated programs that were available to all customers, and informs EfficiencyOne about the participation impacts that result from changes in program delivery. E1’s RBIA model has been the subject of significant...

AI summary The Consumer Advocate highlights flaws in EfficiencyOne's RBIA model, which fails to account for cross-class cost reallocation and avoided costs. They urge the Board to direct NSPI to develop a more accurate RBIA for stakeholder review, emphasizing the need for improved rate-making models in energy efficiency programs.

Page i Date Filed: June 20, 2018 M08604: EfficiencyOne Application approval of 2019 DSM Plan – Reply Submissions 1 1. INTRODUCTION 2 3 EfficiencyOne’s 2019 Demand Side Management Plan (2019 DSM Plan) represents a continuation of the 4 DSM...

AI summary EfficiencyOne's 2019 DSM Plan continuation is supported by stakeholders, with replies addressing intervenor concerns about demand reduction pilots, affordability pilots, and emission estimates. The plan aligns with the Electricity Plan Implementation Act's $34.05M investment cap for 2019.

realized by its current programs (given that they will be 16 designed for demand reduction purposes, rather than energy reduction purposes) and will also generate 17 ancillary energy savings. 18 19 (d) AMI Technology 20 21 The Board has no...

AI summary The document discusses EfficiencyOne's 2019 DSM Plan, emphasizing the need for AMI technology to enable sophisticated demand reduction and the challenges of the Affordable Multifamily Housing Pilot, which has underperformed despite incentives. EfficiencyOne commits to continuing the pilot but highlights the need for cost-effective program design.

9 improvements. This difficulty in realizing project completions has persisted, despite efforts to extend 10 additional support to participants, including incentives of up to 80% of project costs. 11 EfficiencyOne believes it will be more...

AI summary EfficiencyOne faces challenges in completing energy efficiency projects despite 80% cost incentives. It plans to assess the pilot's results by 2018 before deciding on continuation. Discrepancies in emission intensity calculations (0.6531 kg/kWh vs. 0.520 kg/kWh) were highlighted for the 2019 DSM Plan's avoided emissions estimates.

y Reply evidence and submissions by E1 or NS Power were to be filed by June 20, 2018. Please accept this correspondence as NS Power’s reply submission. June 20, 2018 D. Friis Reply Submission NS Power supports the proposed 2019 DSM Plan as...

AI summary NS Power supports the 2019 DSM Plan under EPIA's $34.05M cap, with Synapse endorsing its cost-effectiveness. Intervenors' comments focus on program specifics and transparency, while NS Power suggests a new study for future DSM levels.

ORDER M08604 NOVA SCOTIA UTILITY AND REVIEW BOARD IN THE MATTER OF THE PUBLIC UTILITIES ACT - and - IN THE MATTER OF AN APPLICATION by EFFICIENCYONE for Approval of a Supply Agreement for Electricity Efficiency and Conservation Activities...

AI summary The Nova Scotia Utility and Review Board is considering EfficiencyOne's application for approval of a supply agreement with Nova Scotia Power Inc. and the 2019 Demand Side Management Resource Plan. The process includes a paper hearing, intervenor participation, and a verification review of 2017 program evaluations.

Supply Agreement for Compliance Filing 45 SCHEDULE C 46 47 Performance Requirements 48 1. UARB-APPROVED PERFORMANCE TARGETS. THRESHOLDS. AND 49 INDICATORS 50 51 a) Performance Targets and Thresholds: 52 i. EfficiencyOne is deemed to be in...

AI summary This document outlines the performance requirements for EfficiencyOne under the UARB-approved Plan, including targets and indicators for energy savings, peak demand reduction, ratepayer benefits, and customer satisfaction, with specific reporting obligations and triggers for regulatory processes.

Request IR-6: - Regarding appliance retirements, page 14 states that 386 appliances were replaced, and - 33 footnote 12 states that costs for replacements in non-electrically heated homes are paid - by DSM funds. - a) What was the total co...

AI summary The text contains multiple regulatory requests (IR-6 to IR-11) querying details on appliance retirements, DSM program costs, BNI sector changes, Home Warming program overlaps, HST rebate status, and expenditure surpluses. Questions focus on cost allocation, energy savings, program efficacy, and financial accountability.

Request IR-13: - Page 9 also states that 2019 evaluation activities will be condensed where appropriate in - recognition of the maturity level of some program components. Please identify the - 10 components which will undergo a condensed e...

AI summary Request IR-13 seeks identification of 10 program components whose 2019 evaluations will be condensed due to maturity, aiming to streamline processes while ensuring effectiveness.

Request IR-16: - Regarding the electronic technical reference manual, E1 has stated that it will see several - operational benefits through its use in 2019. In particular, two of those benefits are - identified as decreased evaluation effo...

AI summary The document outlines multiple requests (IR-16 to IR-21) directed at E1, seeking quantification of cost reductions from a technical reference manual, clarification on energy savings targets, material changes in supply agreements with NSPI, and responses to evaluator/verifier recommendations. Key themes include cost evaluation, DSM program performance, and contractual changes.

Request IR-4: - 32 Please refer to Table 2 - Proposed Performance Indicators and Targets on page 15 of the - 33 Evidence. - 34 35 36 a. Please explain why EfficiencyOne does not propose 2019 Performance Targets for lifetime savings, total...

AI summary Request IR-4 questions EfficiencyOne (E1) about missing 2019 performance targets for metrics like lifetime savings and customer satisfaction, whether prior years had such targets, and consequences for unmet thresholds. Request IR-5 demands data on program performance (lifetime benefits, TRC/PAC results) and explains assumptions in the 2019 DSM Resource Plan compared to 2016-2018 plans.

37 Request IR-12: 38 Referring to the 2017 DSM Evaluation Results of Direct Installation program 39 a. Please explain why Direct Installation category appears to include Affordable 40 Multifamily Housing Pilot results in the Small Business...

AI summary The text requests an explanation for the inclusion of Affordable Multifamily Housing Pilot results under the Direct Installation category in the Small Business Energy Solutions portfolio, citing the 2017 DSM Annual Report and requesting supporting documentation such as a UARB Order.

RBIA) by September 30, 2018. 7. E1 is directed to initiate a new DSM Potential Study for completion no later than July 31,2019, to assess the availability of cost-effective DSM measures. 8. E1 is directed to keep the Board apprised of the...

AI summary The Board directs E1 to complete a DSM Potential Study by 2019, improve GHG estimation accuracy, incorporate CO2e emissions into cost-effectiveness analysis, and enhance RBIA and eTRM transparency. E1 must also provide alternate DSM budget scenarios and adhere to standardized filing frameworks, with NSPI conducting rate impact analyses.

Supply Agreement for Compliance Filing 45 SCHEDULE C 46 47 Performance Requirements 48 1. UARB-APPROVED PERFORMANCE TARGETS. THRESHOLDS. AND 49 INDICATORS 50 51 a) Performance Targets and Thresholds: 52 i. EfficiencyOne is deemed to be in...

AI summary This document outlines the performance requirements for EfficiencyOne under the UARB-approved Plan, including specific targets and indicators for energy and peak demand savings, ratepayer benefits, customer satisfaction, and low-income program participation.