E-3ENSC 2011 DSM Evaluation Report prepared by Econoler
126 passages
LIST OF TABLES Table 1: Type(s) of Evaluation Conducted per Program ....................................................................... 1 Table 2: 2011 Savings Targets, Tracked Savings and Evaluated Results for Each Program ..............
AI summary The document lists tables related to program evaluations, savings targets, performance metrics, free-ridership analysis, and recommendations for energy efficiency programs. It outlines data tracking, evaluation methodologies, and general recommendations for specific initiatives like appliance retirement and low-income renter programs.
13.6% 12.8% In 2010, the free-ridership level for CFLs sold through the RMP was integrated with the spillover in a global NTGR. Ref.: 5725 16 Evaluation of 2011 DSM Programs Efficiency Nova Scotia Corporation ECONOL:R Overall Executive Sum...
AI summary The document highlights the need for improved program manuals for Efficiency Nova Scotia Corporation (ENSC) to enhance documentation, data management, and program evaluation. Econoler identified missing components in the current manuals and provided recommendations for improvement.
valuation plan can be tailored to the specifics of a given program. However, a certain amount of basic information is typically included in evaluation plans. The table below presents such information. Table 12: Information Typically Contai...
AI summary The text outlines the typical components of an evaluation plan for a program, including the program description, marketing strategy, partners, evaluation window, objectives, and metrics for monitoring and realizing program targets.
participants or units Objectives monitoring (tracked savings) Data from the program tracking system Objectives realization rate (%) Ratio between targets and tracked savings Business case Details on expected gross and net savings calculati...
AI summary The document outlines key metrics and evaluation criteria for energy efficiency programs, including objectives monitoring, realization rates, business case calculations, and types of evaluations required. It emphasizes factors such as baseline energy levels, technical distortions, free-ridership, and spillover effects in calculating net savings.
viewed the unitary savings estimates, using the metering data from the ARet program and the theoretical energy consumption of the new models installed, based on the ENERGY STAR-qualified product list. As for the process evaluation of the A...
AI summary The evaluation of the Appliance Replacement Program (ARep) highlights the importance of offering financing options to overcome barriers to purchasing large quantities of appliances. Retailers expressed high satisfaction with the program, but noted that upfront payment requirements limited participation. Recommendations include improving access to upfront capital for landlords.
EP-DI-R1. Improve the tracking sheet to allow for a better validation of savings: Several fields in the tracking sheet are not required for program administration. Those fields are present even though they are not filled and should be remo...
AI summary The document recommends improving the EP-DI program's tracking sheet by removing unnecessary fields and ensuring required fields are consistently filled. It also suggests enhancing marketing strategies, as most participants learned about the program through direct interactions rather than marketing materials, and there is a desire for more effective advertising.
nformation for these categories and the establishment of adjustment values that could be used to correct savings. 4.1.8 EnerGuide for Existing Houses Program The 2011 evaluation of the EEH program demonstrated that it is a good energy effi...
AI summary The EnerGuide for Existing Houses (EEH) program in 2011 achieved energy and demand savings, with 1,494 participants. Evaluation highlighted program success but noted concerns about future continuation due to uncertainty around federal government involvement and inconsistent marketing efforts.
Efficiency Nova Scotia Corporation ECONOL:R Overall Executive Summary 4.1.11 Performance Plus Program The 2011 evaluation of the Performance Plus program demonstrated that the program is well designed. The program totalled 701 participants...
AI summary The 2011 evaluation of the Performance Plus program showed it is well-designed, with significant participation growth and notable energy and demand savings. Builders were identified as key partners, and participants were satisfied with the program. Recommendations aim to further optimize its performance.
BER-R2. Inform participants about expected customer savings: Once the final rebate is disbursed to clients, Econoler recommends informing them about the approximate savings to be expected from the project, in terms of energy and money. Thi...
AI summary The document outlines several recommendations for improving the BER program, including informing participants about expected savings, increasing promotion efforts, and collecting additional information to better calculate energy savings from various efficiency measures.
s found in the ARet program manual. Table 4: Program Manual Content Program Manual Content ARet Program Manual Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives...
AI summary The document outlines the content of the ARet Program Manual, which includes sections on program description, incentives, eligibility criteria, cost-effectiveness tests, marketing and evaluation plans, and program theory. It also references a 2011 evaluation report by Efficiency Nova Scotia Corporation.
Program 1. Provide a “base case”: The program manual does not provide a “base case” Manual that provides information on how the assumed gross savings have been Content established. For refrigerators and freezers, they are calculated on a c...
AI summary The document highlights the need for the program manual to include a 'base case' for calculating gross savings, document parameters used for estimating program objectives, and refine and disseminate marketing plans. It emphasizes the importance of transparency, validation, and updating of methodologies and assumptions for accurate forecasting and evaluation.
s, even if they are prepared in another department, should cover a whole year of activities and be included in the program manual. The plans should also identify for each activity the performance indicators that should be collected for mon...
AI summary The document emphasizes the importance of comprehensive program manuals that cover annual activities and include performance indicators for monitoring and evaluation. It also highlights the need for standardization and regular updates to ensure consistency and clarity, especially regarding program details such as the retirement of both primary and secondary units.
Program 9. Develop an evaluation plan in parallel with program design and Design and development: This program has no evaluation plan. Econoler strongly Implementation recommends that for each new program, an evaluation plan be prepared in...
AI summary The text highlights the need for an evaluation plan for a program, the importance of strengthening marketing strategies, communication about appliance disposal, and the pursuit of metering for refrigerators and freezers at the recycling facility to improve program effectiveness and participant engagement.
s there a process flow chart for that program? The web site explains all the steps but there is no illustration. Q: was there an evaluation plan associated to that program when it was first designed? Q: Nowhere do we see the role of the de...
AI summary The document includes a series of questions and comments related to the evaluation and implementation of the Efficient Products – Appliance Retirement Program by Efficiency Nova Scotia Corporation. Questions focus on program design, evaluation plans, delivery processes, and barriers to customer participation.
fectiveness of the program design and delivery.” The process evaluation relied upon in-depth interviews conducted with the PM and with the program partner (retailer). 3.2 PROGRAM MANUAL CONTENT The ARep program features a manual that conta...
AI summary The evaluation of the Appliance Replacement Program (AREP) includes a process evaluation based on interviews with the Program Manager and a program partner. The program manual contains essential components such as program description, objectives, incentives, eligibility criteria, and marketing plans, tailored to internal use by staff and evaluators.
bles such as operating time, wattage or base consumption. The identification and value of each of these variables should be available and documented for validation and updating purposes. > Provide the parameters used to estimate the progra...
AI summary The text discusses the need for documenting variables such as operating time and wattage for validation, and the importance of estimating program objectives using parameters based on past evaluations and market intelligence. It also highlights the need to refine and disseminate marketing plans with performance indicators for monitoring and evaluation.
year of activities and be included in the program manual. For each activity, the plans should identify the performance indicators to be collected for monitoring and evaluation purposes. > Complete the program manual to keep track of change...
AI summary The document discusses the need to complete and standardize the program manual for tracking activities and performance indicators. It also reviews the structure and content of the ARep tracking sheet, noting its usefulness in monitoring refrigerator installations and replacements.
mportant to the PM are covered. The two most important components of the plan are the evaluation methodology as well as the pre- and post-data required to apply the selected methodology. > Focus on a resource acquisition objective: Having...
AI summary The evaluation of the Appliance Replacement Program emphasizes the importance of the evaluation methodology and data collection. The program is focused on resource acquisition rather than market transformation due to its rebate structure and limited advertising. There were challenges with the transition from Nova Scotia Power to Efficiency Nova Scotia Corporation, causing confusion and delays.
rogram manual should document these parameters so that the PM and the Evaluator can keep track of the assumptions used to estimate the program objectives. The parameters should then be adjusted annually based on the most recent evaluation...
AI summary The document discusses the need to document parameters in the program manual to track assumptions used in estimating program objectives and the importance of refining and disseminating annual marketing plans that include performance indicators for monitoring and evaluation. It references an evaluation report for the Appliance Replacement Program by Efficiency Nova Scotia Corporation.
Program 5. Complete the program manual to keep track of changes and standardize Manual the content: The program manual is well written but important information is Content missing to serve as “program memory” in case of modifications or st...
AI summary The document outlines recommendations for improving the program manual and tracking sheet for the Appliance Replacement Program. It suggests standardizing the manual, updating it with recent forms, and adding a field to validate ENERGY STAR qualifications in the tracking sheet to ensure compliance and accuracy.
fficult task. To be discussed with PM. Q: Do you have the source and detail related to the 1608 kWh for the old fridge (this is without degradation factor) so in theory it should be higher? Page 59 C: In figure 2-2 we think that the bubble...
AI summary The text includes a series of questions raised during an evaluation of the Appliance Replacement Program by Efficiency Nova Scotia Corporation. Topics include program name changes, participant confusion, evaluation methods, and the effectiveness of the program's design. Questions also address data sources, evaluation plans, and challenges in implementation.
al. Table 6: Program Manual Content Program Manual Content RMP Program Manual Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives Eligibility criteria Not Applica...
AI summary The document outlines the content of the RMP Program Manual, including sections like program description, incentives, eligibility criteria, and budget. Econoler has evaluated the manual and provided recommendations for improvement.
Efficiency Nova Scotia Corporation 2011 Evaluation Report should document these parameters so that the PM and the Evaluator can keep track of the assumptions used to estimate the program objectives. Moreover, the parameters should be conti...
AI summary The document outlines recommendations for improving program management and evaluation processes, including documenting parameters, refining marketing plans, and completing the program manual. It also mentions the need for more detailed tracking sheets for program administration and evaluation.
noler has identified potential means of improving program design and implementation. The following are comments and recommendations that would have an impact on the design and delivery of the program: > Develop an evaluation plan in parall...
AI summary Econoler suggests improving program design and implementation by developing an evaluation plan in parallel with program development and standardizing the program name across all channels to avoid confusion and ensure consistent evaluation and tracking.
Program 11. Standardize the program name across all channels: To avoid confusion, Tracking Sheet and to ensure adequate evaluation and tracking, the program name should always be the same whether it is for administrative reasons for ENSC (...
AI summary The document discusses the need to standardize the program name for ENSC's energy efficiency initiative, analyze market offers to adjust product discounts, and consider exit surveys to evaluate program effectiveness. It highlights the importance of consistency in branding and the need for data-driven adjustments to ensure program success.
pies of the interview guides and the survey questionnaire are included in Appendices IV, V and VI. Figure 2: Methodological Model 2.2 METHODOLOGY DESCRIPTION 2.2.1 Analyzing Existing Program Information The first task in the process evalua...
AI summary The evaluation of the Low Income Renter (LIR) program involved analyzing program documentation, conducting interviews with the Program Manager, and planning interviews with program partners. The evaluation focused on program justification, savings calculations, barriers, marketing strategies, and the roles of partners.
Efficiency Nova Scotia Corporation 2011 Evaluation Report The methodology used for the on-site visits under the LIR program is further explained in the program impact evaluation section of this report. A copy of the site visit protocol is...
AI summary This section of the 2011 Evaluation Report discusses the methodology used for on-site visits and telephone surveys conducted under the Low Income Renter (LIR) program. A survey of 70 participants was conducted to assess program impact, satisfaction, and recommendations for improvement.
ually included in program manuals intended for internal use. The table below lists the information found in the LIR program manual. Table 4: Program Manual Content LIR Program Program Manual Content Manual Document revisions N/A Program de...
AI summary This document outlines the content of the LIR Program manual, including program description, objectives, eligibility criteria, program partners, savings calculations, budget, cost-effectiveness tests, market studies, marketing plans, and evaluation plans. It is part of an evaluation report from 2011 by Efficiency Nova Scotia Corporation.
Cost-effectiveness test results Market study or technical study that supports program elements or assumptions Marketing plan Evaluation plan Program theory Logic model Program process flow chart Partner or participant process flow chart Pr...
AI summary Econoler evaluated the Low Income Renter (LIR) program manual and found it contains useful information but recommends improvements, such as documenting the disposal of old products and providing guidelines for energy source usage and lighting product installation.
Efficiency Nova Scotia Corporation 2011 Evaluation Report > Provide a “base case”: The program manual does not provide a “base case” that provides information on how the assumed gross savings have been established. Usually, these savings a...
AI summary The 2011 Evaluation Report highlights the need for a 'base case' in the program manual to document assumptions used for calculating gross savings. It also emphasizes the importance of documenting parameters used to estimate program objectives and standardizing the program manual to ensure consistency and ease of use.
more, the content of all program manuals should be standardized based on the same template so that it is easier to locate specific sections or subjects. 3.3 PROGRAM TRACKING SHEET Econoler reviewed the content of the tracking sheet for the...
AI summary The report discusses the need for standardizing program manuals and reviews the tracking sheet for the Low Income Renter (LIR) program. It highlights the tracking sheet's functionality, including unique project identification numbers and energy savings calculations, but notes missing customer information and inconsistencies in product categorization.
type, power and number of old lamps. This information could be used for validating the baseline case and for ensuring that the new lamps installed actually replaced less efficient lamps. > Ensure that all customer information is filled in...
AI summary The text discusses the importance of systematically collecting customer information for program evaluation, such as names, postal codes, and phone numbers, to support activities like customer surveys and on-site visits. It also highlights the success of the Efficient Products – Low Income Renter Program, which had 4,920 participants and benefited from collaboration between ENSC and housing authorities.
Efficiency Nova Scotia Corporation 2011 Evaluation Report > Develop an evaluation plan in parallel with program design and development: This program has no evaluation plan. Econoler strongly recommends that for each new program, an evaluat...
AI summary The evaluation report highlights the need for an evaluation plan to be developed in parallel with program design and development, and recommends consistency in program naming to avoid confusion. The report also notes that program partners are satisfied with their relationship with ENSC’s team.
es and phone numbers. This information is essential for conducting evaluation activities such as customer surveys and on-site visits. In the future, ENSC should ensure that these fields are filled in for all customers. 7. Develop an evalua...
AI summary The document highlights the need for ENSC to ensure complete customer information for evaluation activities and recommends developing an evaluation plan in parallel with program design. It also emphasizes the importance of using a consistent program name to avoid confusion.
cent of the CFLs installed through the program. Nevertheless, the on-site visits did not allow Econoler to confirm that 23 watt CFLs had been used to replace old incandescent lamps with an equivalent light output. Since participants could...
AI summary The evaluation report for the Efficient Products – Low Income Renter Program highlights issues with data collection during on-site visits, including the inability to confirm wattage replacements and the need for more visits to accurately assess installation rates. It also recommends collecting information on air conditioning use for future evaluations.
On a scale from 1 to 10 (where 1 is very bad and 10 is excellent), how would you rate your relationship with the other program partners or subcontractors (Clean Nova Scotia) in this program? Score: Q9a - If your answer to Q9 was equal to o...
AI summary The document contains survey questions related to the Efficient Products – Low Income Renter Program delivered by Efficiency Nova Scotia Corporation. It asks about program partner relationships, lead sources, market confusion, barriers to implementation, program name changes, and complaints received.
on 2011 Evaluation Report 3 PROCESS EVALUATION 3.1 OBJECTIVE AND APPROACH OF THE PROCESS EVALUATION The process evaluation conducted under the MURR program focuses on one of ENSC’s evaluation objectives, which is “assessing the effectivene...
AI summary The process evaluation of the MURR program under ENSC assesses the effectiveness of program design and delivery. The evaluation relied on interviews with the PM and program partners, and a telephone survey of participants. The current program manual is limited due to the program being a pilot, with more detailed content planned for an official launch.
details of gross savings and hypothesis for free- ridership and other effects) Details about the savings calculations and type of products Budget per year Ref.: 5725 6 Efficient Products – Multi-Unit Residential Renter Program Efficiency N...
AI summary The document discusses the need for an in-depth program manual for the Multi-Unit Residential Renter Program, highlighting the current lack of detailed content such as cost-effectiveness test results, marketing plans, and evaluation plans. Econoler has proposed recommendations for improving the manual if the program is officially launched.
bles such as operating time, wattage or base consumption. The identification and value of each of these variables should be available and documented for validation and updating purposes. > Provide the parameters used to estimate program ob...
AI summary The text discusses the importance of documenting program variables and parameters for validation and updating, as well as the need to refine and disseminate marketing plans for program tracking. It references the Efficient Products – Multi-Unit Residential Renter Program and its 2011 Evaluation Report.
d again in the future, Econoler recommends collecting the participants' rate codes in order to establish the line loss factor in a more precise way. 3.4 PROGRAM DESIGN AND IMPLEMENTATION The MURR program was launched in 2011 as a pilot pro...
AI summary The document discusses the MURR program, launched in 2011 as a pilot, and highlights the lack of an evaluation plan. Econoler recommends creating an evaluation plan during program design to ensure data collection and rigorous evaluation. The delivery agent reports satisfaction with the program and no major barriers, though challenges arose from the program's short duration and timing.
at keeps the records? Q: Do you plan to do a post evaluation cost effectiveness test? Q: since this a new program, did you have an evaluation plan designed at the same time the program was designed? Q: Will there be a detailed program mark...
AI summary The text includes a series of questions from an evaluation process for an energy efficiency program, focusing on program design, evaluation plans, marketing strategies, and database usability. It also provides instructions for completing a questionnaire for program partners.
2011 Evaluation Report 3 PROCESS EVALUATION 3.1 OBJECTIVE AND APPROACH OF THE PROCESS EVALUATION The process evaluation conducted under the EP-DI program focuses on one of ENSC’s evaluation objectives, which is “assessing the effectiveness...
AI summary The 2011 Evaluation Report discusses the process evaluation of the EP-DI program, focusing on assessing the effectiveness of program design and delivery. The evaluation involved interviews with the Program Manager (PM) and the Director of Administration (DA), as well as a telephone survey of participants. The report also outlines the content of the program manual, including components such as program description, objectives, incentives, eligibility criteria, and savings calculations.
riers Total savings objectives (with details of gross savings and hypothesis for free-ridership and other effects) Details about the savings calculations and type of products Ref.: 5725 8 Efficient Products – Direct Install Program Efficie...
AI summary The document discusses the Efficient Products – Direct Install (EP-DI) Program, highlighting the content of its program manual, including budget, cost-effectiveness, marketing plans, and evaluation strategies. Econoler's analysis notes that the manual provides useful information for program monitoring, though it lacks specific indicators for measuring marketing success.
the marketing outreach section of the program manual does not specify indicators to measure the success of different efforts, these efforts are sufficiently described for further monitoring purposes. However, the program manual is missing...
AI summary The program manual for the Efficient Products – Direct Install Program lacks specific indicators to measure the success of marketing efforts and does not provide a 'base case' for validating assumed gross savings. Recommendations include documenting program characteristics and providing detailed variables for calculating savings.
Efficiency Nova Scotia Corporation 2011 Evaluation Report > Provide the parameters used to estimate the program objectives: After estimating the gross savings from the base case established in the program manual, the objectives are then fo...
AI summary The document outlines the need to refine parameters for estimating program objectives, incorporating distortion and interactive effects, and emphasizes the importance of maintaining and standardizing the program manual to ensure consistency and track changes over time.
tion, water heater blankets, power bars with timers and occupancy sensors were also piloted in 2011. Econoler has the following comments and recommendations on the design and delivery of the program: > Develop an evaluation plan in paralle...
AI summary Econoler recommends developing an evaluation plan in parallel with program design and standardizing the program name across all channels. The program currently lacks an evaluation plan and has inconsistent naming, which may cause confusion.
2. Provide a “base case”: The program manual does not provide a “base case” that details information on how the assumed gross savings have been established. Usually, these savings are calculated with an algorithm using a series of variable...
AI summary The document highlights the need for a 'base case' in the program manual to establish gross savings through documented variables. It also emphasizes the importance of including parameters for estimating program objectives, accounting for distortion and interactive effects, and updating the manual to maintain consistency and track changes over time.
usion between representative and delivery agent. Page 7 C: Make sure that replaced lamps where functional before replacing them (they were not burned) to get an accurate calculation of the savings. C: Interactive effects will have to be ca...
AI summary The text discusses the Efficient Products – Direct Install Program by Efficiency Nova Scotia Corporation, focusing on program evaluation, delivery agents, quality assurance, and process documentation. It includes questions about program structure, participant forms, barriers addressed, and evaluation plans.
interviews conducted with the PM and with the program partners, through on-site visits with participants as well as through a telephone survey of program participants. 3.2 PROGRAM MANUAL CONTENT The EEH program features a manual that conta...
AI summary The EnerGuide for Existing Houses (EEH) program includes a program manual with components such as program description, objectives, incentives, eligibility criteria, and savings calculations. The manual is a critical tool for program staff and evaluators, and is tailored to meet specific organizational needs.
Efficiency Nova Scotia Corporation 2011 Evaluation Report > Provide the parameters used to estimate the program objectives: After estimating the gross savings from the base case established in the program manual, the objectives are then fo...
AI summary The document outlines the need to refine program objectives by incorporating distortion and interactive effects, standardize marketing plans with performance indicators, and improve the program manual to serve as a comprehensive and standardized reference for tracking changes and program memory.
d for each prescriptive measure are available in the program tracking sheet. However, Econoler recommends adding this information in the program manual. 2. Provide the parameters used to estimate the program objectives: After estimating th...
AI summary The text discusses the need to document parameters used for estimating program objectives, including distortion and interactive effects, and the importance of refining and disseminating marketing plans within the program manual for effective monitoring and evaluation.
gram manual. The plans should also identify for each activity the performance indicators that should be collected for monitoring and evaluation purposes. Ref.: 5725 67 EnerGuide for Existing Houses Program Efficiency Nova Scotia Corporatio...
AI summary The document highlights the need for a comprehensive program manual for the EnerGuide for Existing Houses Program, emphasizing the importance of standardizing content and including performance indicators for effective monitoring and evaluation.
age of missing entries for the EnerGuide rating is quite low for 2011 (only 3%), the Evaluator cannot provide ENSC with gross energy savings for these projects. Program 10. Develop an evaluation plan in parallel with program design and Des...
AI summary The Evaluator cannot provide ENSC with gross energy savings for 2011 due to a low rate of missing EnerGuide ratings. Econoler recommends developing an evaluation plan in parallel with program design and suggests strengthening partnerships with DAs by involving them in program modifications and EE measure decisions.
re it changed status and name from Conserve Nova Scotia to ENSC, how was this transition perceived in the market? Not applicable Unnoticed Some confusion A lot of confusion Don’t know Ref.: 5725 91 EnerGuide for Existing Houses Program Eff...
AI summary The text discusses the transition of the program from Conserve Nova Scotia to ENSC and its impact on program partners, including changes in perception, relationship with the Program Administrator, and participant complaints. It also includes questions about the ease of participation in the EnerGuide for Existing Houses Program.
7. Develop an evaluation plan in parallel with program design and Pilot Design and Implementation development: This pilot has no evaluation plan. Econoler strongly recommends that for each new pilot or program, an evaluation plan be prepar...
AI summary The text emphasizes the importance of developing an evaluation plan alongside program design, providing real examples in program materials, and considering partnerships with delivery agents and subcontractors to ensure effective program implementation and control.
2011 Sample Size 28 Male 57% Female 43 Ref.: 5725 33 Fuel Substitution Pilot Efficiency Nova Scotia Corporation 2011 Evaluation Report APPENDIX IV - INTERVIEW GUIDE WITH PROGRAM MANAGER Process Evaluation Program Name: Fuel Substitution Pi...
AI summary This document contains an interview guide for the Fuel Substitution Pilot Program managed by Efficiency Nova Scotia Corporation (ENSC). It includes questions and comments related to the program's implementation, such as equipment disposal, incentive calculations, and the clarity of program materials.
notice that natural gas equipment have no capacity specified. Care to comment? Q: The incentives have been calculated based on what exactly: uninstalling, acquisition, installation and inspection? Ref.: 5725 34 Fuel Substitution Pilot Effi...
AI summary The text contains a series of questions and comments regarding the Fuel Substitution Pilot program managed by Efficiency Nova Scotia Corporation (ENSC). Topics include program incentives, participant responsibilities, form requirements, and verification processes for eligibility and cost calculations.
nuals intended for internal use. The table below presents such information as well as that found in the Low Income program manual. Table 4: Program Manual Content Low Income Program Manual Content Program Manual Document revisions Program...
AI summary The document outlines the content of program manuals, specifically focusing on the Low Income Program, including sections such as program description, eligibility criteria, budget, and evaluation plans. It references an evaluation report from Efficiency Nova Scotia Corporation in 2011.
ons Marketing plan Evaluation plan Program theory Logic model Program process flow chart Partner or participant process flow chart Program forms Information included in the program manual Information partially included in the program manua...
AI summary Econoler evaluated the Low Income program manual and found it useful but recommended revisions to the process flow chart, inclusion of EnerGuide audit and trainer activities, and monitoring of outreach activities with performance indicators.
vities: The Evaluator recommends that outreach activities be monitored with performance indicators to evaluate the effectiveness of each activity in terms of enrolment in the program. Ref.: 5725 8 Low Income Program Efficiency Nova Scotia...
AI summary The Evaluator recommends monitoring outreach activities with performance indicators to assess program effectiveness. The program manual lacks a 'base case' for calculating gross savings and does not document parameters used to estimate program objectives, which should be based on past evaluations and adjusted annually.
and the Evaluator can keep track of the assumptions used to estimate the program objectives. The parameters should then be adjusted annually based on the most recent evaluation results. > Complete the program manual to keep track of change...
AI summary The text discusses the need to improve the program manual for tracking changes and standardizing content, as well as the structure of the Low Income program tracking sheet. It highlights the importance of annual adjustments based on evaluation results and the need for better association between project and client information for easier tracking and evaluation.
and documented for validation and updating purposes. 5. Provide the parameters used to estimate the program objectives: After estimating the gross savings from the base case established in the program manual, the objectives are then foreca...
AI summary The text discusses the methodology for estimating program objectives in energy efficiency programs, emphasizing the use of parameters based on past evaluations, market intelligence, and other sources. It also highlights the importance of documenting these parameters to ensure transparency and annual adjustments based on the latest evaluation results.
Pay Electric Bills 2011 Sample size 70 Yes 100% Ref.: 5725 64 Low Income Program Efficiency Nova Scotia Corporation 2011 Evaluation Report APPENDIX IV - INTERVIEW GUIDE WITH PROGRAM MANAGER Process Evaluation Program Name: Low Income House...
AI summary This document is an evaluation report from 2011 focusing on the Low Income Household program managed by Efficiency Nova Scotia Corporation. It includes an interview guide with the program manager to assess the status of various recommendations and changes to the program, such as the name change from REAP and the inclusion of appliances like refrigerators and freezers.
program manual they are if they are of a certain age. If they are not included anymore, what is the reason? Q: Scope I are mainly envelope measure and Scope II are prescriptive measures. Correct? Ref.: 5725 65 Low Income Program Efficiency...
AI summary The text includes questions and references related to a low-income program evaluation report by Efficiency Nova Scotia Corporation. Topics include program scope, funding responsibilities, and the status of recommendations. There are also inquiries about energy audit processes and program modifications.
elivery agent who trains the energy auditors or NRCan? Q: IN the program objectives, the reduction of arrears on their electricity bill is one of them? Page 7 The logic model needs to be modified: 1. In the program activity line- there sho...
AI summary The text discusses modifications to a logic model for a low-income energy program, including training of delivery agents, audit processes, and referrals. It also references program name changes and qualifying measure updates over time.
e Household Program 4. And now, Low Income Homeowner Care to comment? Q: We understand that the qualifying measures have changed from last year, correct? Source Program Manual Page 3 C: It is not easy to read who does what in the introduct...
AI summary The text discusses feedback on the Low Income Homeowner Program, including suggestions for improving clarity in the program manual, concerns about budget management, and questions about protocols for replacing and disposing of CFLs. It also references an evaluation report and mentions the need for performance indicators and process evaluations.
his form within the next 48 hours, save the file and send it back at the following email address: [email protected]. Thank you! Part I: General Questions Please indicate your company name: Q1 - How would you define your role(s) as an E...
AI summary This document contains general questions directed to ENSC energy efficiency program partners, focusing on their role, experience with the transition from Conserve Nova Scotia to ENSC, and their relationship with the Program Administrator. It also asks about complaints received from program participants.
m Information The first task in the process evaluation of the Performance Plus program was the analysis of the program documentation. At this stage, all available program information was reviewed. Ref.: 5725 4 Performance Plus Program Effi...
AI summary The evaluation of the Performance Plus program began with a review of program documentation and an in-depth interview with the Program Manager, covering topics such as program justification, savings calculations, and marketing strategies. Additional interviews with program partners, including builders, were conducted as part of the evaluation process.
cy concepts and potential measures for reducing energy consumption (such as the free guide “Keeping the Heat In”). Since residential participants often have a low understanding of the proposed energy efficiency measures, such informational...
AI summary The document discusses the importance of improving communication in energy efficiency programs, emphasizing the need for better information on recommended measures and increasing program advertising. It highlights that participants often have a low understanding of energy efficiency measures and that program marketing should focus on energy bill savings and rebates as key motivators for participation.
43 70 Male 8 63% 69% Female 2 37 31 Ref.: 5725 Performance Plus Program Efficiency Nova Scotia Corporation 2011 Evaluation Report APPENDIX III - INTERVIEW GUIDE WITH PROGRAM MANAGER Process Evaluation Program Name: Performance Plus (New Ho...
AI summary The text discusses the Performance Plus Program evaluation report from 2011, including interview questions with the program manager regarding the program's status, free-ridership, baseline standards for new construction, and the status of various recommendations.
e Guide Q: On the Web site we see only incentives to electricity measures. In the program manual we have both incentives coming from rate-payer funds and coming from provincial funds, Why is that? Ref.: 5725 Performance Plus Program Effici...
AI summary The document contains questions and clarifications regarding the Performance Plus Program, including the source of funding, program name changes, and the involvement of multiple stakeholders. Questions focus on incentives, program implementation, and data collection practices.
collect performance indicators (PI) for all those activities? Page 11 – table 4.3.1 Q: Do you use deemed savings for the measures described in pages 5 to 8? Or do you calculate them individually? Q: If they are deemed savings do you have a...
AI summary The text discusses questions and comments regarding the Performance Plus Program, including the use of deemed savings, the need for updated variables, and the adequacy of program procedures. It also highlights issues with the participant form, such as the need for clarity, revision dates, and the inclusion of utility names.
inspected. - The scoring mechanism to calculate the free-ridership percentage from the three questions included in the application form should be documented in the program manual. Ref.: 5725 7 Business Energy Rebates Program Efficiency Nov...
AI summary The document discusses the need to document the scoring mechanism for calculating free-ridership percentage in the Business Energy Rebates Program and the importance of refining and disseminating marketing plans. It emphasizes the need for program manuals to include parameters for estimating program objectives and performance indicators for monitoring and evaluation.
year of activities and be included in the program manual. For each activity, the plans should identify the performance indicators to be collected for monitoring and evaluation purposes. > Complete the program manual to keep track of change...
AI summary The document discusses the need to update and standardize the program manual for the Business Energy Rebates (BER) program, including performance indicators for monitoring and evaluation. It also mentions the review of the BER program tracking sheet and the use of online documents to validate project details.
tors and Drives category is identified in three different ways. This will facilitate the site sampling process and the savings validation once a systematic validation can be conducted. Ref.: 5725 10 Business Energy Rebates Program Efficien...
AI summary The Business Energy Rebates Program, managed by Efficiency Nova Scotia Corporation, has seen significant growth in participation, with 106 participants in 2011 compared to 13 in the first year. The program was upgraded with streamlined application processes, market push strategies, and free-rider screening. Econoler provided recommendations for further improvements.
o be noted that since only one on-site visit was conducted for this category, and considering the unexpected nature of the findings, no further adjustment was applied to savings for the HVAC category. The Evaluator recommends that the PM c...
AI summary The evaluation report highlights the need for the Program Manager (PM) to confirm equipment usage with participants before using default full load hours to avoid overestimating savings. It also recommends increasing on-site visits for HVAC and motor/drives measures to improve accuracy in savings calculations. A commissioning factor was not applied due to lack of evidence that commissioning occurred.
Program Manual 1. Document important program assumptions or practices: Content – The Evaluator understands that the measures included in the program as well as the gross savings they generate are based on a similar program run by EVT. Rega...
AI summary The document discusses the need for a comprehensive program manual that includes assumptions, practices, and parameters for estimating program objectives. It emphasizes documenting savings calculations, incentive structures, project selection criteria, and free-ridership scoring mechanisms for accuracy and transparency in program evaluation.
rms of net savings. To that effect, the program objectives include any distortion effects, whether positive or negative. They also include interactive effects, when not taken into account in the base case. These parameters are usually base...
AI summary The text discusses the importance of documenting program parameters and refining marketing plans for the Business Energy Rebates program. It emphasizes the need to account for distortion and interactive effects in program objectives and to use performance indicators for monitoring and evaluation.
0-day delay to process the application and receive the incentive should be monitored to ensure that it is respected and avoid any complaints regarding delays. 12. Inform participants about expected customer savings: Once the final rebate i...
AI summary The text outlines recommendations for improving the Business Energy Rebates (BER) program, including monitoring application processing delays, informing participants of expected savings, including revision dates on application forms, and emphasizing energy bill savings in program communications.
an you comment on that recommendation? BER-R1 What is the status of this recommendation about the free-riders (46%) is it the 3 questions you ask in the application form available from your web site? BER-R2 Are you using more contractors a...
AI summary The document discusses the Business Energy Rebates program, including questions about free-ridership, application processes, data collection, and the handling of non-admissible incentive requests. It also references an evaluation report and application forms.
s that are not qualified for the incentive? Q: If yes what is the percentage of these requests that are not admissible? Q: Do you keep a record of these non-admissible requests per type of measure? FAQ C: It’s a good practice to confirm pr...
AI summary The text discusses questions and comments regarding the Business Energy Rebates program, including eligibility criteria, application processing times, inspection practices, and the clarity of the program manual. Concerns are raised about the admissibility of incentives, communication with participants, and the accessibility of the program documentation.
calculations are done with the algorithms imbedded in the database, but our question is: Will the participant eventually know what will be the savings that he should expect from each measure? Page 8 Q: The table of incentives for each meas...
AI summary The document discusses concerns about the clarity and usability of the Business Energy Rebates program manual, including repeated sections, missing form elements, and the need for simplification. Questions are raised about who fills out the worksheets and where the data comes from.
ny or organization to purchase energy-efficient products when installing or replacing energy-using products for your business in the future? [98 = Don’t know/Don’t recall, 99 = Refused] S3. Did you experience any specific challenge or face...
AI summary The text contains survey questions related to the Business Energy Rebates (BER) program administered by Efficiency Nova Scotia Corporation (ENSC), focusing on participant experiences, challenges, and suggestions for improvement. The survey aims to gather feedback on program barriers, issues encountered, and potential enhancements.
lues for the fields listed above. Nevertheless, these invoices were approved by the PM. The missing information might not be essential for program administration but is useful for evaluation purposes. For each sale, the information in the...
AI summary The evaluation of the Smart Lighting Choices Program highlights that while invoices contain necessary information for rebate administration, additional details such as product specifications are missing. The Program Manager uses product numbers to determine lamp wattage and ballast types for tracking savings. The Evaluator suggests improvements to invoice consistency and program evaluation.
consumer)? C: Since the end users are unknown (beside large customers) sales records of a previous year from retailer could give an identification of the program impact in the market. Source: Web Q: When referring to the CEE Web Site, whic...
AI summary The text discusses the Smart Lighting Choices Program managed by Efficiency Nova Scotia Corporation, including questions about program participation, reporting forms, and the need for clearer separation of sections in the program manual for participants, retailers, and ENSC. It also references an interview guide for distributors.
med 25 on-site visits as well as a telephone survey of 27 program participants. The analysis and program evaluation results are presented separately for each of the two C&I Custom program components. PROGRAM OVERVIEW The C&I Custom program...
AI summary The C&I Custom programs offer technical assistance, financial incentives, and project financing to medium and large C&I customers to reduce energy consumption and demand. The C&IC program component, launched in 2008, was upgraded in 2011 with improved staffing, focus on larger projects, and enhanced sales outreach to improve peak demand savings.
m a participation point of view, these are indications that program marketing efforts are improving from past years, which means that the program clearly evolved in the right direction. Satisfaction Satisfaction towards the C&IC program ha...
AI summary The C&IC program has seen improved satisfaction, particularly in staff interaction and communication, following a transition from NSPI to ENSC. Despite some concerns about the M&V process, overall satisfaction remains high. Energy savings were calculated through on-site visits and interviews, with adjustment factors used to estimate overall program savings.
Corporation 2011 Evaluation Report 3.2 METHODOLOGY DESCRIPTION 3.2.1 Analyzing Existing Program Information The first task in the process evaluation of the C&I Custom program was an analysis of the program documentation. At that stage, the...
AI summary The evaluation of the C&I Custom program involved analyzing program documentation and conducting in-depth interviews with Program Managers to assess program justification, savings calculations, barriers, marketing strategies, and prior evaluation recommendations.
participants is presented in Appendix VI. 4 The sampling population for the 2011 survey was 68 participants, which was the number of program participants at the time of conducting the survey. Ref.: 5725 11 Commercial & Industrial Custom an...
AI summary The process evaluation of the Commercial & Industrial Custom and New Construction (C&IC) program focuses on assessing the effectiveness of program design and delivery. The evaluation involves examining the program manual, which serves as an essential reference tool for program staff and was widely used by the evaluator during the process and impact evaluation.
program manuals intended for internal use. More specifically for the C&IC program, the table below presents such typical program manual contents along with what was found in the C&IC program manual. Table 4: C&IC Program Manual Content Eva...
AI summary The document evaluates the content of the C&IC program manual, highlighting typical sections such as program description, objectives, eligibility criteria, and program partners. It notes that certain sections, such as total savings objectives and market studies, are either missing or partially included in the manual.
s multiple purposes such as determining project eligibility, establishing the level of incentive, estimating free-ridership levels, etc. This should be explained in the program manual. > Clarify the use of the terms “preliminary energy aud...
AI summary The text discusses the need for clarity in terminology related to energy audits and scoping studies, and the importance of documenting parameters used for estimating program objectives. It emphasizes avoiding confusion between terms and ensuring transparency in assumptions for future evaluations.
Furthermore, many “Superseded” subfolders could be found in the project folders. Such subfolders are used for storing old versions of electronic files, which could be useful to track project history. Prior to January 1, 2011, each project...
AI summary The document discusses the management of project folders and the evolution of project numbering systems used by NSPI and ENSC. It highlights the need for standardized nomenclature for electronic files and project names to improve consistency and program monitoring.
e marketing activities should be enhanced even if the verification partner expressed a high level of satisfaction with the marketing and outreach activities. 4.3 CINC PROGRAM PROCESS EVALUATION This section presents the key outcomes from t...
AI summary The document discusses the evaluation of the Commercial & Industrial Custom and New Construction (CINC) program, focusing on the process evaluation conducted by the Evaluator. It highlights the program manual content and compares it with typical details found in program manuals. The evaluation involved interviews with the Program Manager and partners, as well as a participant.
11 Evaluation Report Table 5: CINC Program Manual Content Evaluation Program Manual Content CINC Program Manual Document revisions Program description (including nature or type of program) Program justification Program objectives Incentive...
AI summary The document evaluates the CINC program manual, identifying areas for improvement. It highlights missing sections such as total savings objectives, savings calculations, and market studies. The evaluation report recommends enhancing the manual's structure and content for better clarity and completeness.
rack of the assumptions used to estimate the program targets. After each evaluation, these parameters are adjusted accordingly for estimating the program objectives of the years to come. > Standardize the use of terms in Section 4.3.3: In...
AI summary The evaluation report discusses the need for standardizing terminology in the program manual and providing a printable version of the CPG for participants. It also highlights the need for additional information to serve as 'program memory' and standardization of program manuals based on a common template.
arison could help identify which one is closest to reality. It was mentioned that ABI would do this comparison for the projects undertaken under the NCCP path when data are available. Ref.: 5725 26 Commercial & Industrial Custom and New Co...
AI summary The document discusses the need for Efficiency Nova Scotia Corporation (ENSC) to establish performance indicators to evaluate the efficiency of the project delivery process under the Commercial & Industrial Custom and New Construction Program (C&I C&NC Program). It references time limits outlined in the program manual and suggests using these to monitor task completion.
evaluating project delivery process efficiency, ENSC should establish performance indicators to monitor if such tasks are completed within the time frame predicted in the program theory. > Clearly define the CPG minimum requirements at the...
AI summary The Evaluator recommends that ENSC establish performance indicators to monitor project delivery efficiency and provide standardized minimum requirements for CINC projects early in the process. Additionally, the Evaluator suggests adding a commissioning component to the program to help achieve predicted energy savings.
Section of 15. Supplement the ENSC website with information taken from Appendix H of ENSC Website the C&IC program manual: Appendix H of the program manual, intended to facilitate staff training, is a very good initiative in case there are...
AI summary The text discusses suggestions for improving the ENSC website and program awareness, including using Appendix H for staff training and FAQs, changing section titles to avoid confusion, and enhancing marketing to increase participation by emphasizing financial benefits.
Program 13. Develop an evaluation plan in parallel with program design and Design and development: This program has no evaluation plan. The Evaluator strongly Implementation recommends that for each new program, an evaluation plan be prepa...
AI summary The text emphasizes the importance of developing an evaluation plan alongside program design, comparing actual building consumption with simulations for accuracy, and setting performance indicators for project delivery. These steps aim to improve program effectiveness and ensure accurate monitoring and verification.
h when data are available. 15. Set performance indicators on project delivery process: In different sections of the program manual there are references to time limits to perform a given task. With the aim of evaluating project delivery pro...
AI summary The document recommends establishing performance indicators for project delivery efficiency and clarifying minimum requirements for CINC projects to improve compliance and reduce issues during the M&V step. It also suggests providing standardized documentation and examples to assist participants in meeting CPG requirements.
Efficiency Nova Scotia Corporation 2011 Evaluation Report Page 2 Q: If you use required internal standard as a base line for some cases, would it still be that level even if its lower than the (MNECB) 97 requirements? Page 3 (M& V) Q: If y...
AI summary The document contains questions and comments regarding the Evaluation Report of Efficiency Nova Scotia Corporation from 2011. Issues raised include baseline standards, data validation, program structure, financial calculations, and participant support materials. The discussion focuses on improving data collection, program clarity, and participant engagement.
interested in internal procedures. Q: Do you have a participation flow chart that illustrates the different steps for the participants that could be a good support for the program manual writing? Ref.: 5725 80 Commercial & Industrial Custo...
AI summary The text includes questions and comments related to program eligibility, internal procedures, terminology consistency, time limits, and program funding caps in the Commercial & Industrial Custom and New Construction Program. Concerns are raised about free-ridership, clarity of documentation, and potential barriers caused by funding limitations.
cts because of that limitation? Section 4.4.2.2 C: The 85% ratio for incentive of the total program budget seems a very good ratio for a new program. I have to congratulate you if this is the case. Q: I understand that you do not allow red...
AI summary The text includes questions and comments regarding program design, payback periods, commissioning plans, legal reviews, training materials, marketing strategies, program logic models, and potential distortion effects such as free-riders and spillovers. These discussions pertain to the evaluation and structure of the Commercial & Industrial Custom and New Construction Program.
Care to comment? Q: In the appendix A, we count eleven (11) forms for that program. Could that be a reason for any bottleneck in tracking the program data and entering them in a centralized database? C: Custom Client Response Process (afte...
AI summary The text discusses questions raised during a regulatory proceeding regarding program data tracking, methodology appendices, marketing strategies, and program evaluation. It highlights concerns about bottlenecks, missing appendices, and the need for evaluation plans and cost-effectiveness testing.
tion report) and we hope that they will contribute to improving program performance. If we need to clarify some of your answers, we might call you back. Your name Date Phone number E-mail Ref.: 5725 95 Commercial & Industrial Custom and Ne...
AI summary This document is a process evaluation questionnaire for the Custom Program (Existing Commercial and Industrial) under the ENSC Energy Efficiency Program. It asks participants to provide feedback on their role, relationship with the Program Administrator and Project Coordinator, and other general questions.
program barriers; > program marketing and outreach strategy; > participation forms and flow charts; > role of the program partners; and > prior evaluation recommendations. Based on the results of the PM interview, the list of questions and...
AI summary The evaluation of the Small Business Energy Solutions (SBES) program involved interviews with program partners and on-site visits to validate the implementation of energy efficiency measures. These activities aimed to assess program barriers, marketing strategies, and participant satisfaction.
as well as the associated sampling error at the 90 percent confidence level for the 2008, 2009, 2010 and 2011 telephone surveys. Table 3: Sample Size and Sampling Error Sampling Error at 90% Survey Population Sample Size (n) Confidence Int...
AI summary The process evaluation of the Small Business Energy Solutions Program focuses on assessing the effectiveness of the program's design and delivery. It was conducted through interviews with the Program Manager and Delivery Agents, as well as a telephone survey of program participants.
am performance. This process evaluation was performed through interviews with the PM and with the DAs, as well as through a telephone survey of program participants. 3.2 PROGRAM MANUAL CONTENT The SBES program features a manual that contai...
AI summary The evaluation of the SBES program involved interviews with the PM and DAs, as well as a telephone survey of participants. The program manual includes components such as program description, objectives, incentives, eligibility criteria, and cost-effectiveness test results, tailored for internal use by program staff and evaluators.
Efficiency Nova Scotia Corporation 2011 Evaluation Report SBES Program Program Manual Content Manual Market study or technical study that supports program elements or assumptions Marketing plan Evaluation plan Program theory Logic model Pr...
AI summary Econoler evaluated the SBES program manual and found it contains useful information on responsibilities, eligibility criteria, and administrative forms. However, it lacks critical information that makes understanding the program design and delivery difficult, with some issues being minor and others more significant.
to be inspected should be explained in the program manual. Also a standard inspection report could be designed and used as a performance indicator for management and evaluation purposes. > Complete the program manual to keep track of chang...
AI summary The text emphasizes the need for a standardized program manual to track changes and serve as a reference for program management and evaluation. It highlights the importance of updating program manuals, providing two examples related to the SBES program, including a name change and modifications to vendor selection processes.
nsure that the information is systematically entered in these fields. Conversely, the fields that are not used should be removed from the tracking sheet in order to focus on useful data. > Make sure that all fields are systematically fille...
AI summary The document emphasizes the importance of systematically filling in all fields on the tracking sheet to ensure accurate data collection. It highlights that certain fields, such as building type and proposed hours of operation, had significant missing data in the 2011 tracking sheet, which could impact future evaluations and savings validation.
Efficiency Nova Scotia Corporation 2011 Evaluation Report > Set limitations for entries and develop a validation process: To ensure that incorrect values are not entered in the tracking sheet, a validation process should be developed by EN...
AI summary The 2011 Evaluation Report discusses modifications to the SBES program, including the introduction of multiple material vendors, new lighting controls, and a participant process flow chart to improve program implementation and energy savings.
ogram without affecting other programs will generate additional savings. Finally, ENSC developed a participant process flow chart in 2011 in order to make the program participation procedures clearer. ENSC is planning to change the program...
AI summary ENSC is planning to update the SBES program in 2012 by expanding beyond lighting measures. The Evaluator recommends replacing current DAs with more versatile organizations and developing an evaluation plan in parallel with program design to ensure rigorous evaluation and better program outcomes.
the SBES program: > Consider replacing current DAs with organizations not limited to lighting specialization. > Develop an evaluation plan in parallel with program design and development. Ref.: 5725 13 Small Business Energy Solutions Progr...
AI summary The SBES program suggests replacing current DAs with more diversified organizations and developing an evaluation plan alongside program design. The ENSC website provides information about the SBES program, including eligibility, costs, and enrollment processes. Partners reported minimal confusion during the transition from NSPI to ENSC.
Accept faxed contracts: One DA suggests accepting faxed contracts as opposed to originals to improve performance. ENSC could also develop a web-based contract as opposed to paperwork. Ref.: 5725 14 Small Business Energy Solutions Program E...
AI summary A DA suggests accepting faxed contracts to improve performance and streamline eligibility and financing procedures for the SBES program. The program's eligibility criteria and potential for scaling through franchise participation are discussed.
t suggested improvements to the SBES program. Accelerating the process was also suggested by five percent of respondents. Over half of the 2011 respondents could not provide an improvement suggestion. The following are participant comments...
AI summary The 2011 evaluation report of the Small Business Energy Solutions Program highlights the need for improvements in marketing, communication, and program clarity. Participants emphasized the lack of awareness and difficulties in understanding audit results and the slow approval process.
of variables such as operating time, wattage or base consumption. The identification and value of each of these variables should be available and documented for validation and updating purposes. 5. Provide the parameters used to estimate t...
AI summary The text discusses the documentation and validation of variables such as operating time and wattage in energy efficiency programs, as well as the parameters used to estimate program objectives, including distortion and interactive effects. The program manual should document these parameters for tracking assumptions and annual adjustments based on evaluation results.
Program 6. Make explicit plans for post-installation inspections: The criteria or basis for Manual selecting projects to be inspected should be explained in the program manual. Content Also a standard inspection report could be designed an...
AI summary The document outlines recommendations for improving program manuals, including post-installation inspections, standardization of content, and updating program names and vendor policies. It emphasizes the need for a standardized template and continuous updates to ensure clarity and effective program management.
Program 10. Determine the required fields and remove the unnecessary ones: The PM Tracking together with the Evaluator should determine which fields are necessary for Sheet administration and evaluation purposes and ensure that the informa...
AI summary The text discusses the need to refine a program tracking sheet by removing unnecessary fields, ensuring systematic data entry, and implementing validation processes. It highlights issues with missing and inconsistent data in the 2011 tracking sheet, particularly in fields like building type, bill account number, and proposed hours of operation.
- INTERVIEW GUIDE WITH PROGRAM MANAGER Process Evaluation Program Name: Small Business Energy Solutions Telephone Interview with the Program Manager Date: List of questions (Q) and comments (C) discussed Source: Evaluation Report 2010 Page...
AI summary This text discusses the evaluation of the Small Business Energy Solutions (SBES) program, including questions about the program's theory, incentives, structure, and eligibility criteria. It also mentions the program's name change from Small Business Direct Install and considerations for improving the program manual.
ng a minimum number of installations? Page 7 Q: If the main objective of ENSC is to save electricity, why are municipal utilities not eligible for the bill financing of 20% like the clients of NSP? Q: What is the proportion (%) of particip...
AI summary The text contains a series of questions raised during a regulatory proceeding regarding the Small Business Energy Solutions Program operated by Efficiency Nova Scotia Corporation. The questions focus on program eligibility, incentive levels, verification processes, and administrative procedures.
satisfaction (say from 1 to 10) for most of the questions instead of a straight yes or no or not sure. C: Replace question # 3, by if the score is under .8, ask for the reason(s). Source: General Q: Could you please describe who the progra...
AI summary The text outlines a process evaluation questionnaire for Efficiency Nova Scotia Corporation's Small Business Energy Solutions Program, aimed at gathering feedback from program partners regarding their roles, program design, barriers, cost-effectiveness testing, marketing strategies, and database usability.
t applicable It was the same Program Administrator I had to deal with a new PA The transition went smoothly The transition caused some delays or adjustments Other, please comment Q2 - On a scale from 1 to 10 (where 1 is very bad and 10 is...
AI summary The text is a survey related to the Small Business Energy Solutions Program administered by Efficiency Nova Scotia Corporation. It asks participants about their experience with the Program Administrator, satisfaction with the program, complaints received, and suggestions for improvement.
lved in the delivery of that program? Yes: If yes, from which party? (Please do not mention names, just the position of the party.) No Don’t know Q14a- What was the complaint about? Ref.: 5725 54 Small Business Energy Solutions Program Eff...
AI summary The text is part of an evaluation report for the Small Business Energy Solutions Program by Efficiency Nova Scotia Corporation. It includes a questionnaire for electricians and program partners to identify barriers to program implementation and gather feedback for improving performance.
rogram, did you experience or witness any confusion in the market among the parties, including yours and/or the participants? Yes: which ones? What was the confusion about? No Don’t know Ref.: 5725 59 Small Business Energy Solutions Progra...
AI summary The text is from an evaluation report of the Small Business Energy Solutions Program by Efficiency Nova Scotia Corporation. It includes questions about program name changes, modifications, complaints, and barriers to implementation. The report seeks feedback on the program's operation and effectiveness.
Efficiency Nova Scotia Corporation 2011 Evaluation Report Q8 – Last year, have you been interviewed by another evaluator about the same program? Yes No Not applicable Q8a – If you answered Yes to Q8, we would like your opinion on the frequ...
AI summary This document is part of an evaluation report for the Small Business Energy Solutions Program conducted by Efficiency Nova Scotia Corporation in 2011. It includes questions about program evaluation, partnerships, market confusion, and program modifications.
120102012 DSM Evaluation Reports
46 passages
3.1 PROGRAM MANUAL Program manuals feature key information on program process and tracking for internal use and are an essential reference tool for all program staff. In addition, program manuals are widely used by the Evaluator throughout...
AI summary The program manual is a critical tool for internal program staff and evaluators, capturing key program information. The 2012 version improved upon the 2011 format and followed a major evaluator recommendation, with a standardized template across all ENSC programs. Econoler recommends further improvements to supplement the tool.
3.2 DATA TRACKING Data tracking and reporting are crucial for program management as well as for evaluation purposes. In 2010, ENSC implemented an online Demand-Side Management Data System (DSMDS), and has been working on its improvement si...
AI summary Data tracking and reporting are critical for program management and evaluation. ENSC implemented the DSMDS in 2010 but it had limitations, particularly in generating comprehensive reports. Econoler used tracking sheets for the 2012 evaluation and noted improvements since 2011, though challenges remain in validating savings calculations. Recommendations were made to improve data consistency and support the DSMDS development.
The 2012 evaluation of HEA demonstrated that the program underwent a series of changes that altered its design and implementation. In addition to the withdrawal of the federal government from the program, modifications were made to the for...
AI summary The 2012 evaluation of the Home Energy Assessment (HEA) program showed significant changes in design and implementation, including the removal of mechanical heating system rebates and the introduction of new tools for program management. Partners noted a lack of collaboration from Natural Resources Canada (NRCan) and issues with the HOT2000 software.
Table 24: Recommendations for Business Energy Rebates No. Recommendations BER-R1. Analyze product offering to reduce duplicate rebates and program overlapping in the market: Some eligible measures covered by the BER instant rebate approach...
AI summary The text discusses the need to analyze and adjust the Business Energy Rebates (BER) program to avoid duplicate rebates and program overlap with other BNI programs like BES and Custom Retrofit. It highlights that 37% of T8 lamps and 48% of ballasts eligible for BER instant rebates are already covered by other programs, suggesting a need to limit BER rebates to products with low market penetration, such as LED technologies.
3.2 PROGRAM DOCUMENT ANALYSIS AND INTERVIEW WITH THE PROGRAM MANAGER The following section describes the Evaluator's key findings and recommendations resulting from the analysis of several program documents and the interview with the PM. I...
AI summary The Evaluator highlighted the importance of having a program memory and evaluation plan for ARet. In 2011, the lack of standardized program memory was a key recommendation. By 2012, ARet had implemented a program manual and an evaluation plan based on the Evaluator's template.
APPENDIX I - RECOMMENDATIONS Sections Recommendations 7. Document major changes in the Document Revision section: In the Document Revision section, it is advisable to give a brief description of the major changes incorporated into the new...
AI summary The recommendation suggests updating the Document Revision section of the program manual to briefly describe major changes, such as the exclusion of dehumidifiers and the addition of AIR MILES® rewards, to provide readers with a quick overview of revisions without needing to compare old and new versions.
Program Charts 11. Provide partners and participants with simplified process charts: The Evaluator recommends developing a simplified participant process chart and posting it on the program website, which is comprehensive, clear and well d...
AI summary The text discusses the need for simplified participant process charts and more quantitative performance indicators in outreach activities. It emphasizes clarity, accessibility, and regular updates to ensure effectiveness and informed decision-making.
Q7a. Did you receive any complaints from other program partners? No Yes : If yes, from which partners? (Please do not mention names, just the party's role) Q7b. What was the complaint/were the complaints about? Q8. What suggestions do you...
AI summary The text presents a form used to collect feedback from program partners regarding complaints and suggestions for improving program performance. It includes questions about complaints received, their nature, and suggestions for enhancing outreach and marketing efforts.
Awareness and Marketing In order for the program to keep being successful in the years to come, program awareness constitutes an important aspect of the program strategy. Survey results showed that nearly six in ten CFL or LED buyers had h...
AI summary The Instant Savings program's success depends on effective awareness and marketing strategies. Surveys show high customer recognition of the program, with in-store promotions and customer engagement events contributing significantly. However, challenges remain, including difficulties in engaging store personnel and tight timelines for campaign preparation. Retailers also suggest expanding the range of eligible products to increase program impact.
3.2 PROGRAM DOCUMENT ANALYSIS AND INTERVIEW WITH THE PROGRAM MANAGER The following section describes the Evaluator's key findings and recommendations resulting from the analysis of several program documents and the interview with the PM. I...
AI summary The section discusses the Evaluator's findings and recommendations from analyzing program documents and interviewing the Program Manager. Key recommendations included implementing a program memory and developing evaluation plans. Progress was made in 2012, with the program memory included in the manual and an evaluation plan for Instant Savings.
Table 4: Program Manual Content – Instant Savings Program Manual Content 2011 2012 Document revisions 4 4 Program description (including nature or type of program) 4 4 Program justification Δ 4 Program objectives 1 1 Incentives 4 4 Eligibi...
AI summary Table 4 outlines the content of the Instant Savings program manual across different years, detailing sections such as program description, incentives, eligibility, and evaluation plans. It shows the inclusion or absence of various elements in the manual for 2011 and 2012.
3.3 TRACKING SHEET Econoler reviewed the content of ENSC's tracking sheet for Instant Savings. The tracking sheet for 2012 is very clear and useful. All recommendations from 2011 were incorporated into the 2012 tracking sheet, which makes...
AI summary Econoler reviewed ENSC's tracking sheet for the Instant Savings program, noting that the 2012 version was comprehensive and incorporated all 2011 recommendations. The tracking sheet includes detailed sales data, product information, and rebate discounts, though some inconsistencies were found for certain products due to manual rebate applications.
Executive Summary 5. Improve the content of the tracking sheet : Econoler's review of the tracking sheet showed that ENSC took into consideration the recommendations from last year's evaluation and the Evaluator concludes that the 2012 tra...
AI summary Econoler's review of the tracking sheet for the Instant Savings program found it to be complete and consistent, with recommendations for future improvements. It also suggested that the Document Revision section of the program manual should briefly describe major changes, such as the addition of the AIR MILES® component in certain stores.
Q7b. What was the complaint/were the complaints about? Q8a. Now, approximately how many complaints have you received from retailers thus far in 2012? Number of complaints: Q8b. What was the complaint/were the complaints about? Q9. What sug...
AI summary The text contains a series of questions posed to retailers regarding complaints received in 2012, the nature of those complaints, and suggestions for improving program performance, particularly in marketing and outreach. The questions are part of a confidential evaluation process aimed at improving program effectiveness.
3.2 PROGRAM DOCUMENT ANALYSIS AND INTERVIEW WITH THE PROGRAM MANAGER The following section describes the Evaluator's key findings and recommendations resulting from the analysis of several program documents and the interview with the PM. I...
AI summary The section outlines the Evaluator's findings and recommendations from program document analysis and an interview with the Program Manager. Key recommendations from 2011 included implementing a program memory and developing evaluation plans. These were partially addressed in 2012, with the program memory included in the manual and HEA having an evaluation plan.
Table 4: Program Manual Content - HEA Program Manual Content 2011 2012 Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives Eligibility criteria Program partners a...
AI summary Table 4 outlines the content of the Program Manual for the Heat Energy Assistance (HEA) program, detailing sections such as program description, eligibility criteria, incentives, and evaluation plans. The table shows that most content was not included in the manual for the years 2011 and 2012, with only 'Program forms' noted as 'N/A'.
This Appendix presents all recommendations made by the Evaluator throughout the report as well as the sections where the recommendations originated. Sections Recommendations Program Manual 4. Document major changes in the Document Revision...
AI summary This appendix compiles all recommendations made by the Evaluator, including where they originated in the report. It focuses on documentation practices, specifically the need to describe major changes in the Document Revision section.
Number of complaints: Q7b. What was the complaint/were the complaints about? Q8. What suggestions do you have to improve program performance? Q9. More precisely, what suggestions do you have concerning program marketing and outreach activi...
AI summary The text includes questions about the nature of complaints, suggestions for improving program performance, and specific recommendations for marketing and outreach activities. It also mentions confidentiality and the potential for follow-up contact.
The new version of the program manual is a considerable improvement from the 2011 version. For additional improvements, the Evaluator has a few minor recommendations. - > Document the major characteristics of the program: In the program ma...
AI summary The document discusses improvements to a program manual, noting that the 2011 version needs updates. The Evaluator recommends documenting incentives offered by Heritage Gas for converting to natural gas and detailing major changes in the Document Revision section, such as the program's transition from a pilot to a full program and its shift toward promoting green energy sources like heat pumps and biomass.
3.2.4 Program Charts The program now also has two process charts, one for the participant and the other for the program manager. These charts were recommended in the 2011 evaluation and are very informative for any new staff responsible fo...
AI summary The program now includes two process charts for participants and program managers, recommended in the 2011 evaluation. Econoler suggests posting a simplified participant process chart on the program website to ensure clarity and wide distribution for maximum exposure.
DEFINITIONS Base case A base case details the information on how assumed gross savings used in the tracking sheet have been established. Usually, these savings are calculated with a series of variables such as hours of operation, wattage o...
AI summary The text defines key terms related to energy efficiency programs, including demand savings, distortion effects, free-ridership, and interactive effects. It also explains metrics such as the demand-to-energy ratio and line loss factor, which are used in evaluating program performance and energy savings.
various places in the present report, which are aimed at optimizing the program. Based on the evaluation results, the most important recommendations that Econoler has for this program are as follows: - PP-R1. Provide participants with more...
AI summary The report outlines recommendations to improve the program, including providing more information on recommended measures, simplifying the incentive table, and supporting DAs in marketing efforts. These changes aim to enhance program effectiveness and participant engagement.
2.2.2 Interviews with Program Manager Econoler staff conducted two interviews in July and September 2012 respectively with the Program Manager (PM). These interviews were intended, among other things, to follow up on the 2011 evaluation re...
AI summary Econoler conducted interviews with the Program Manager in 2012 to follow up on 2011 evaluation recommendations and understand changes in program delivery. These interviews informed the preparation of data collection tools for a process evaluation.
3.2.1 Program Manual A program manual is an important tool for the entire program staff as well as for the Evaluator. Indeed, it serves as a program memory and can be particularly useful in case of staff changes. The purpose of the program...
AI summary The Program Manual is a standardized tool used by program staff and evaluators to document program details, objectives, and changes. It is structured into 10 sections and may reference other documents to keep the manual concise.
Table 6: Program Manual Content – Performance Plus Program Manual Content 2012 Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives Eligibility criteria Program pa...
AI summary Table 6 outlines the content of the Program Manual for the Performance Plus initiative, detailing sections such as program description, objectives, incentives, eligibility, and evaluation plans. It includes a list of program manual content items for the year 2012, indicating what information is included, partially included, or missing.
CONCLUSION The 2012 evaluation of Performance Plus demonstrated that the program underwent a series of changes that altered its design and implementation. Indeed, Performance Plus went from attributing distinctive incentives for every impl...
AI summary The 2012 evaluation of the Performance Plus program showed significant improvements since 2011, including better program design, increased promotion, and higher participant satisfaction. The program saw an increase in participants and achieved higher energy and demand savings. However, program partners noted that more marketing and information on the assessment process were needed.
IN3. On a scale from 1 to 10 (where 1 is very poor and 10 is excellent), how would you rate your satisfaction in regard to the…? Aspects of the program Score 1=very poor 10=excellent Reason If you answered 8 or less, please share the reaso...
AI summary The text presents a survey assessing participant satisfaction with various aspects of an energy efficiency program, including program marketing, relationships with ENSC staff, and the assessment process. It includes questions about the appropriateness of the EnerGuide methodology and the need for additional information or changes to the assessment process.
SECTION PM – PROGRAM AND MEASURES The next questions will concern the measures promoted by the Performance Plus program. PM1a. Regarding program on-course modifications, (change in total cost of the assessment, change in incentives for pre...
AI summary The text outlines a series of questions related to the Performance Plus program, focusing on program modifications, barriers to implementation, participant complaints, and suggestions for improvement. It includes inquiries about the program's effectiveness, participant challenges, and areas for enhancement.
2.2.2 Interviews with Program Manager Econoler staff conducted two interviews with the Program Manager (PM), in July 2012 and August 2012 respectively. The main objective of these interviews was to gain a basic understanding of the program...
AI summary Econoler staff conducted two interviews with the Program Manager in July and August 2012 to understand program mechanisms and delivery, which informed the development of data collection tools for the process evaluation.
Table 4: Program Manual Content Program Manual Content - RDI 2012 Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives Eligibility criteria Program partners and th...
AI summary Table 4 outlines the content of the Program Manual for the Residential Direct Install (RDI) program in 2012, listing various sections such as program description, incentives, eligibility criteria, and evaluation plans, with some sections marked as 'Not Applicable' or lacking information.
[IF ONLY ONE MEASURE VERIFIED IN V3-V4, SKIP TO FR4] The next questions will be about the [MEASURE 2] you had installed through the program. FR1M2. Did you or anyone in your household have specific plans to purchase and install [MEASURE 2]...
AI summary The text outlines a series of questions aimed at determining whether participants in the Residential Direct Installation Program had pre-existing plans to install [MEASURE 2] and assessing the likelihood they would have taken similar actions without the program.
Executive Summary 3. Collect quantity of products installed in the tracking sheet for each individual tenant unit\ : In 2012, the quantity of products installed in each individual unit was only collected on paper forms from mid-August. In...
AI summary The text discusses recommendations for improving data collection and tracking sheets in energy efficiency programs. It suggests collecting product installation data by unit number instead of tenant names to respect privacy and improve landlord communication. It also recommends enhancing the tracking sheet with additional fields for better evaluation of program impacts and savings.
Marketing and Outreach Activities 9. Make the performance indicators more quantitative: To make the current performance indicators in the marketing plan more precise, specific and effective, they should be modified to reflect more quantita...
AI summary The text discusses improvements to marketing performance indicators and data collection in a program. It recommends making performance indicators more quantitative, ensuring all fields in the tracking sheet are filled, and collecting additional data such as CFL installation location and heating/air conditioning information for better evaluation.
2.2.1 Analyzing Existing Program Information Analyzing program documentation was the first task in the process evaluation for BER. At this stage, all of the available program information was reviewed. The information used to conduct this a...
AI summary The first task in the process evaluation for BER was analyzing program documentation. Information was gathered from sources such as the ENSC website, Program Manual, Evaluation Plan Schedule A CIRx Screening Tool, and Tracking Sheet.
2.2.2 Interviews with Program Manager Econoler staff conducted two interviews with the Program Manager (PM) respectively in July and August 2012. Among other things, these interviews were intended to follow up on the 2011 evaluation recomm...
AI summary Econoler staff conducted interviews with the Program Manager in July and August 2012 to follow up on 2011 evaluation recommendations and understand changes in program delivery in 2012. This information was used to prepare questions for data collection tools for a process evaluation.
3.2 PROGRAM DOCUMENT ANALYSIS AND INTERVIEWS WITH THE PROGRAM MANAGER The following section describes the Evaluator's key findings and recommendations resulting from the analysis of several program documents and the interviews with the PM....
AI summary This section discusses the Evaluator's findings and recommendations from analyzing program documents and interviews with the Program Manager. Key points include the implementation of a program memory in the program manual and the development of an evaluation plan for the Business Energy Rebates program.
4.1 OBJECTIVE AND APPROACH OF THE IMPACT EVALUATION The impact evaluation for mail-in rebate is aimed at determining the gross and net savings of this component of BER for 2012. Both energy and demand savings were considered in the evaluat...
AI summary The impact evaluation for the mail-in rebate component of the Business Energy Rebates (BER) program in 2012 aimed to determine gross and net savings, considering energy and demand savings. The evaluation involved analyzing parameters such as savings methodology, equipment characteristics, and free-ridership, supported by tracking sheets, 50 on-site visits, and participant surveys.
3.1 OBJECTIVE AND APPROACH OF THE PROCESS EVALUATION In 2011, Econoler performed a comprehensive process evaluation of this program. The 2010 evaluation report and recommendations, the program manual and website, the application forms, the...
AI summary This section outlines the objective and approach of the 2012 process evaluation, which is a continuation of the 2011 evaluation. It focuses on following up on previous recommendations, reviewing updated program documentation, and gathering perspectives from consultants and participants.
3.2 PROGRAM DOCUMENTS ANALYSIS AND INTERVIEW WITH THE PROGRAM MANAGER The following section presents the Evaluator's key findings and recommendations resulting from the analysis of several program documents and the interview with the PM. I...
AI summary The Evaluator reviewed program documents and interviewed the Program Manager, highlighting improvements such as the inclusion of a program memory in the manual and the development of an evaluation plan for Custom Retrofit since 2011 and 2012.
Table 4: Program Manual Content – Custom Retrofit Program Manual Content 2011 2012 Document revisions Program description (including nature or type of program) Program justification Program objectives Incentives Eligibility criteria Progra...
AI summary Table 4 outlines the content of the Program Manual for the Custom Retrofit program, including sections such as program description, incentives, eligibility criteria, and evaluation plans. The table compares the content across 2011 and 2012, with some sections marked as 'N/A' for 2011.
Q3. On a scale from 1 to 10, where 1 is very poor and 10 is excellent, please rate your satisfaction in regard to the: Aspects of the program Score 1 = very poor 10 = excellent NA = Not Applicable Reason If you answered 8 or less, please s...
AI summary The document presents a survey with questions about satisfaction levels regarding various aspects of a program, including technical support, processing delays, marketing, and relationships with staff and clients. It also asks about the program's impact on business and barriers to its delivery, along with complaints received.
3 TRACKING SHEET As part of the program evaluation process, Econoler reviewed the structure and contents of the NC tracking sheet. This tracking sheet is actually an extract from the online Demand-Side Management Data System (DSMDS). The E...
AI summary Econoler evaluated the NC tracking sheet, an extract from the DSMDS, and found it to be simple with minimal details. The 2012 version includes DSMDS numbers and NSPI rate codes, but lacks building size figures and project status information compared to the 2011 version.
Adjustments to NC Projects Conducted under the NCEM Approach Each NCEM project was given an adjustment factor established on the basis of the findings and observations from the project document analyses and on-site visits performed. Since...
AI summary The evaluation of NCEM projects in 2012 found that most projects were implemented as expected, but two projects (11% of expected savings) did not yield significant energy savings. One project was delayed due to incomplete construction, and another (1.5% of savings) had incomplete insulation work, making it ineligible.
m the base cases presented in the Evaluation Plan. In the light of these observations, the Evaluator makes the following recommendations to facilitate program monitoring and evaluation in the future:
AI summary The Evaluator provides recommendations to improve future program monitoring and evaluation based on observations from the base cases presented in the Evaluation Plan.
4.1 OBJECTIVE AND APPROACH OF THE IMPACT EVALUATION The impact evaluation for SBES aims at determining the gross and net savings of this component for 2012. Both energy and demand savings were considered in this evaluation. To do so, the f...
AI summary The impact evaluation of the SBES program in 2012 assesses both gross and net energy and demand savings. Key parameters analyzed include unitary wattage values, installation rates, and free-ridership. On-site visits and participant surveys were used to validate data and calculate the net-to-gross ratio (NTGR), which was used to determine net savings. The evaluation also includes savings from non-lighting measures like heat pump installations and two pilot programs.
Tracking Sheet SBES - 5. Develop a validation process for savings calculations: To ensure that the savings recorded in the tracking sheet correspond to those calculated using the parameters filled out for each measure (quantity, unitary wa...
AI summary The tracking sheet SBES requires improvements in validation processes for energy savings calculations, better identification of lighting control installations, and additional fields to distinguish indoor and outdoor measure installations. These changes aim to improve data accuracy and facilitate the evaluation of interactive effects.