View Notice

Evaluation Resource Expert for Stage I HCFC Phase-out Management Plans (HPMPs) 2 Positions
Procurement Process :RFP - Request for proposal
Office :UNDP - UNITED STATES OF AMERICA
Deadline :04-Jun-18
Posted on :14-May-18
Development Area :CONSULTANTS  CONSULTANTS
Reference Number :46384
Link to Atlas Project :
00038474 - Montreal Protocol Management Project
Overview :

Background:

The United Nations Development Programme (UNDP) is one of the four Implementing Agencies (IAs) designated by the Multilateral Fund (MLF) to implement the Montreal Protocol´s Ozone Depletion Substances (ODS) phase-out projects.

UNDP’s ozone and chemicals global programmes are managed by the Montreal Protocol and Chemicals Unit (MPU/Chemicals) within the Sustainable Development Cluster of the Bureau for Policy and Programme Support (BPPS) in New York, with staff also located in UNDP Regional Centers in Istanbul Panama City and Bangkok. Since 1991, UNDP has received over US $800 million in grants from the MLF, the Global Environment Facility (GEF) and bilateral donors to undertake thousands of activities related to ozone and chemicals in nearly 100 developing countries and countries with economies in transition.  Please refer to this link for more information on MPU/Chemicals’ activities on ozone layer protection:http://www.undp.org/content/undp/en/home/librarypage/environment-energy/ozone_and_climate/past-successes-and-future-opportunities--case-studies-from-the-u.html.

The Parties to the Montreal Protocol agreed under the Decision XIX/6 on the "Adjustment to the Montreal Protocol with regard to Annex C, Group I substances (HCFCs)" to "accelerate the phase-out of production and consumption of the hydrochlorofluorocarbons (HCFCs)", and the Article 5 (A5) countries have been requested by the MLF’s Executive Committee to adopt a staged approach to the implementation of their HCFC Phase-out Management Plans (HPMP). 

HPMPs are performance-based agreements between the country and the MLF Executive Committee, whereby agreed-upon funding tranches are released when conditions related to ODS phaseout and disbursements are met.

The preparation of the HPMP included the following elements:

  • Assistance to determine relevant institutional, policy and legislation framework;
  • Survey on HCFC consumption, use and trade;
  • Development of strategies and action plans for compliance with Stage I;
  • Development and finalization of HPMP, including stakeholder consultations and support; and
  • Individual HCFC phase-out investment project proposals.

The implementation of the approved HPMP entails a series of activities for the execution of activities approved under the respective project documents, following the national needs identified during the phase-out preparation period.

UNDP is supporting the implementation of Stage I HPMPs in 47 countries covering both large and small ODS consumers.  We are currently concluding Stage I, which entails meeting the 2013 and 2015 HCFC control measures and starting in 2016, we have initiated Stage II in most countries, which addresses the HCFC control measures through 2020 and for many countries, beyond 2020.

Evaluation Approach, Scope and Methodology:

The objective of the evaluation of Stage I HPMP design and implementation is to analyze and document the results and lessons learned from the funding received by the MLF for countries to develop national strategies to achieve the 2013 freeze and 2015 control target (“Stage I”), to provide recommendations regarding the achievement of the 2020 control target (“Stage II) as well as the upcoming hydrofluorocarbon (HFC) phasedown, and highlight opportunities for scaling up and replicating good practices.  This evaluation will be conducted according to the guidance, rules and procedures established by UNDP and the Handbook on Planning, Monitoring and Evaluating for Results .  

The evaluation must provide evidence‐based information that is credible, reliable and useful. The evaluation resource expert is expected to:

  • Provide support to the lead evaluator and contribute to development of Stage I HPMP framework, review questions, Stage I HPMP evaluation report structure and detailed timeline;
  • Review and synthesize all relevant sources of information, such as the UNDP project document, project reports – including Stage I HPMPs, progress reports, national strategic and legal documents, project completion reports (PCRs), MLF evaluations and any other materials that the evaluator considers useful for this evidence-based assessment; and
  • Provide feedback on draft full Stage I HPMP Evaluation report.

The methodological approach of the evaluation of Stage I HPMPs is expected to include the following main elements:

  • Analysis of all prepared Stage I HPMPs (47 countries’ HPMPs have been approved with support from UNDP by the MLF Executive Committee) to assess their overall quality and relevance, barriers identified and scope for implementation. The analysis should also consider issues around technology selection and implementation methodologies used during Stage I HPMP implementation, reflect on how these lessons can be applied to Stage II HPMP implementation as well as the upcoming hydrofluorocarbon (HFC) phasedown, and highlight opportunities for replication and scaling up of good practices to other countries/regions. 
  • Stakeholder analysis through structured virtual interviews with (a) the MLF Secretariat, regarding the results, operations and management of the UNDP MLF programme; (b) government counterparts, in particular the National Ozone Unit (NOU) Officer; (c) UNDP Country Offices, who are key partners in the implementation of HPMPs; (d) UNDP Montreal Protocol/Chemicals Technical advisers based in the regions, who provide overall advice and guidance on HPMP design and implementation; (e) technical consultants that provide guidance on the development of the Stage I HPMP; and (f) other key stakeholders.
  • Selected thematic analysis and lessons from across Stage I HPMPs, e.g. across key MLF sectors (foam, refrigeration, air conditioning, servicing, etc), Low Volume ODS consuming Countries (LVC) and non-LVCs, technology selection drivers, geographic similarities, cooperation with the private sector, number of jobs created, human health impact, is also included.

Specific Evaluation Criteria:

The five standard evaluation criteria developed by the OECD/DAC for evaluating development assistance (relevance, efficiency, effectiveness, impacts and sustainability) will be applied as outlined in below[1]:

OECD/DAC Evaluation Criteria

Definition[2]

Specific Relevance to Stage I HPMP Evaluation

Relevance

The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies.

·        Are the activities outlined in the country’s national HPMP strategies relevant to address the protection of the ozone layer and the climate?

·        Relevance to and inclusion in national development strategies?

Efficiency

A measure of how economic resources/inputs (funds, expertise, time, etc.) are converted to results.

·        Incurred transaction costs from preparing and implementing HPMPs

·        Comparison of UNDP implementation modalities (NIM vs DIM)

·        UNDP CO role In HPMP implementation

·        Effectiveness of MPU/Chemicals support to countries

Effectiveness

The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.

·        Did countries meet the control targets established under the Stage I HPMP strategy on time?

·        Adequate financing for Stage I HPMP activities secured?

Impacts

Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.

·        Did the Stage I HPMP projects lead to any other positive or negative consequences, in particular the non-environmental effects on productivity and enterprise competitiveness as well as employment effects?

·        Results relating to capacity development in the government as a result of the Stage I HPMP

Sustainability

The continuation of benefits from a development intervention after major development assistance has been completed. The probability of continued long-term benefits. The resilience to risk of the net benefit flows over time.

·        Ownership of the Stage I HPMP and commitment to follow up by governments and other partners of existing and potential future control measures under the Montreal Protocol

Composition of Evaluation Team:

The evaluation team will be composed of one evaluation expert, who will be supported by two key resource experts with technical knowledge and understanding of the Montreal Protocol.  The evaluation expert will be designated as the team leader and will be responsible for finalizing the report and will be supported by the two key resource people. 

 

Evaluation Deliverables and Timeline:

 

The evaluation team is expected to deliver the review according to the following schedule:

Deliverable

Content

Timing

Responsibilities

Inception Report

Evaluator provides clarifications on evaluation methodology that will be used and the list of key questions that will be asked during the evaluation

3 days

 

Submission to UNDP MPU HQ for comments and approval

Presentation

Initial Findings, review/summarize document

15 days

Submission to UNDP MPU HQ for comments and approval

Draft Final Report

Full report, (per annexed template) with annexes

10 days

Submission to UNDP MPU HQ, reviewed and commented on by UNDP MPU RTAs

Final Report*

Revised report

Within 1 week of receiving UNDP comments on draft (2 days)

Submission to UNDP MPU HQ for final approval

*When submitting the final evaluation report, the evaluator is required also to provide an 'audit trail', detailing how all received comments have (and have not) been addressed in the final evaluation report.

Evaluation Ethics:

Evaluation consultants will be held to the highest ethical standards. UNDP evaluations are conducted in accordance with the principles outlined in the UNEG 'Ethical Guidelines for Evaluations'

Institutional Arrangement:

The consultant will report to, seek approval/acceptance of outputs from the task team leader (MPU/Chemicals Director).

The contractor is expected to have her/his own office space, laptop and access to internet and a printer during the assignment (the cost of this facilities should be included in the financial proposals, if required).

Payment terms:

Payments will be made based on the agreed financial proposal and released upon submission of a certificate of payment request, indicating deliverables achieved and days worked to be verified and cleared for payment by the supervisor.

Competencies

  • Demonstrated capacity for strategic thinking;
  • Proven capacity to produce high quality qualitative research and ability to absorb, analyze and synthesize large amounts of complex information within tight deadlines;
  • Demonstrated ability and willingness to work with people of different cultural, ethnic and religious background, different gender, and diverse political views.
  • Demonstrated ability to communicate complex issues in a concise and clear manner
  • Highly organized with strong analytical, synthesis, and research skills

Qualifications of the Successful Individual Contractor:

Education:

  • Master’s degree in natural resources management, engineering, sciences, environment, climate sciences, sustainable development, international development, public policy, social sciences, economics, public administration, finance or other closely related fields.

Experience:

  • At least 10 years of working experience with the Montreal Protocol;
  • At least 5 years of experience in project and programme design and development, project and programme management and implementation, and/or monitoring and evaluation/ knowledge.
  • Experience with international organizations desired, experience with UNDP an added benefit.

Recommended Presentation of Offer:

The application is a two-step process. Failing to comply with the submission process may result in disqualifying the applications:

Step 1: Interested individual consultants must include the following documents when submitting the applications in UNDP job shop (Please note that only 1 (one) file can be uploaded therefore please include all docs in one file):

  • Personal History Form (P11), indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references (the template can be downloaded from this link: UNDP P-11 Form
  • Brief description of why the individual considers him/herself as the most suitable for the assignment. Indicate available start date.

Step 2: Submission of Financial Proposal

Applicants are instructed to submit their daily rate financial proposals in US Dollars for this consultancy to bpps.procurement@undp.org using the financial proposal template available here: http://procurement-notices.undp.org/view_file.cfm?doc_id=45780. The proposals should be sent via email with the following subject heading: “Financial Proposal for Consultant, Evaluation Expert for MPU/Chemicals Programme" by the deadline for this vacancy. Proposals to be received after the deadline may be rejected. In order to assist the requesting unit in the comparison of financial proposals, the financial proposal should be all-inclusive and include a breakdown. The term ‘all-inclusive” implies that all costs (professional fees, travel related expenses, communications, utilities, consumables, insurance, etc.) that could possibly be incurred by the Contractor are already factored into the financial proposal. 

Travel

Travel is not anticipated at this time but may be included at a later date as an addendum to the approved contract with costs based on UNDP travel guidelines. 

               Scope of Price Proposal and Schedule of Payments:

§  Financial proposals must be all inclusive[i] and must be expressed on the basis of “a daily fee” in USD.

§  Payment will be made upon monthly submission of a certificate of payment request, indicating outputs achieved and days worked to be verified and cleared for payment by the supervisor.

Criteria for Selection of the Best Offer:

Only those candidates that meet the minimum level of education and relevant years of experience requirements will be considered for the technical evaluation. The technical evaluation will include a desk review to select the shortlisted candidates. The technical evaluation may also include interviews with shortlisted candidate(s).

The selection of the best offer from the shortlisted candidates will be based on a Combined Scoring method – where the technical evaluation (desk review and interview) will be weighted a maximum of 70 points, and combined with the price offer which will be weighted a maximum of 30 points. The 70 points rating shall be based on how well the Offer or meets the minimum qualifications/competencies described above. 

The technical evaluation will be based on the following criteria with the corresponding points (out of a total 70 points):

Criteria 1:             Master’s degree in natural resources management, engineering, sciences, environment, climate sciences, sustainable development, international development, public policy, social sciences, economics, public administration, finance or other closely related fields; (10 points)

Criteria 2:             At least 10 years of working experience with the Montreal Protocol; (40 points)

Criteria 3:             At least 5 years of experience in project and programme design and development, project and programme management and implementation, and/or monitoring and evaluation/ knowledge; (30 points).

Criteria 4:             Experience with international organizations desired, experience with UNDP an added benefit (20 points)

Only candidates obtaining a minimum of 49 (70%) points on technical evaluation will be considered for the Financial Evaluation.

Financial evaluation (maximum 30 points):

The following formula will be used to evaluate financial proposal:

p = y (μ/z), where

p = points for the financial proposal being evaluated
y = maximum number of points for the financial proposal
μ = price of the lowest priced proposal
z = price of the proposal being evaluated

 

[1] http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm

[2] http://www.oecd.org/dac/evaluation/2754804.pdf

[i] The term ‘all inclusive” implies that all costs (professional fees, communications, utilities, consumables, insurance, etc.) that could possibly be incurred by the Contractor are already factored into the final amounts submitted in the proposal.

 

Annex 1: Evaluation Report Outline

i.

Opening page:

  • Title of evaluation
  • MLF projects covered under evaluation
  • Evaluation time frame and date of evaluation report
  • Region and countries included in the project
  • Implementing Partner and other project partners
  • Evaluation team members
  • Acknowledgements

ii.

Executive Summary

  • Summary of conclusions, recommendations and lessons learned from Stage I HPMP design and implementation

iii.

Acronyms and Abbreviations

(See: UNDP Editorial Manual[i])

1.

Introduction

  • Purpose of the evaluation
  • Scope & Methodology
  • Structure of the evaluation report

2.

Description and development context

  • Start and duration of Stage I HPMP implementation
  • Problems that the projects sought  to address
  • Immediate and development objectives of the projects
  • Baseline Indicators established
  • Main stakeholders
  • Expected Results

3.

Findings

(In addition to a descriptive assessment, all criteria marked with (*) must be rated[i])

3.1

Design / Formulation

  • Assumptions and Risks
  • Planned stakeholder participation
  • Replication approach
  • UNDP comparative advantage
  • Technology choices
  • Linkages between project and other interventions within the sector
  • Management arrangements

3.2

Implementation

  • Adaptive management (changes to the project design and project outputs during implementation)
  • Partnership arrangements (with relevant stakeholders involved in the country/region)
  • Feedback from M&E activities used for adaptive management
  • Project Finance 
  • Monitoring and evaluation: design at entry and implementation (*)
  • UNDP and Implementing Partner implementation / execution (*) coordination, and operational issues

3.3

Results

  • Overall results (attainment of objectives)
  • Relevance
  • Effectiveness & Efficiency
  • Sustainability
  • Impact

4.

Conclusions, Recommendations & Lessons

  • Recommended actions to improve the design, implementation, monitoring and evaluation of Stage I HPMPs
  • Actions to follow up or reinforce initial benefits from the projects
  • Proposals for future directions underlining main objectives
  • Best and worst practices in addressing issues relating to relevance, performance and success

5.

Annexes

  • ToR
  • Itinerary (if travel is involved)
  • List of persons interviewed
  • Summary of field visits (if travel is involved)
  • List of documents reviewed
  • Evaluation Question Matrix
  • Questionnaire used and summary of results
  • Evaluation Consultant Agreement Form
  • Report Clearance Form 

 

 

 

Annex 1: Evaluation Report Outline

i.

Opening page:

  • Title of evaluation
  • MLF projects covered under evaluation
  • Evaluation time frame and date of evaluation report
  • Region and countries included in the project
  • Implementing Partner and other project partners
  • Evaluation team members
  • Acknowledgements

ii.

Executive Summary

  • Summary of conclusions, recommendations and lessons learned from Stage I HPMP design and implementation

iii.

Acronyms and Abbreviations

(See: UNDP Editorial Manual[i])

1.

Introduction

  • Purpose of the evaluation
  • Scope & Methodology
  • Structure of the evaluation report

2.

Description and development context

  • Start and duration of Stage I HPMP implementation
  • Problems that the projects sought  to address
  • Immediate and development objectives of the projects
  • Baseline Indicators established
  • Main stakeholders
  • Expected Results

3.

Findings

(In addition to a descriptive assessment, all criteria marked with (*) must be rated[i])

3.1

Design / Formulation

  • Assumptions and Risks
  • Planned stakeholder participation
  • Replication approach
  • UNDP comparative advantage
  • Technology choices
  • Linkages between project and other interventions within the sector
  • Management arrangements

3.2

Implementation

  • Adaptive management (changes to the project design and project outputs during implementation)
  • Partnership arrangements (with relevant stakeholders involved in the country/region)
  • Feedback from M&E activities used for adaptive management
  • Project Finance 
  • Monitoring and evaluation: design at entry and implementation (*)
  • UNDP and Implementing Partner implementation / execution (*) coordination, and operational issues

3.3

Results

  • Overall results (attainment of objectives)
  • Relevance
  • Effectiveness & Efficiency
  • Sustainability
  • Impact

4.

Conclusions, Recommendations & Lessons

  • Recommended actions to improve the design, implementation, monitoring and evaluation of Stage I HPMPs
  • Actions to follow up or reinforce initial benefits from the projects
  • Proposals for future directions underlining main objectives
  • Best and worst practices in addressing issues relating to relevance, performance and success

5.

Annexes

  • ToR
  • Itinerary (if travel is involved)
  • List of persons interviewed
  • Summary of field visits (if travel is involved)
  • List of documents reviewed
  • Evaluation Question Matrix
  • Questionnaire used and summary of results
  • Evaluation Consultant Agreement Form
  • Report Clearance Form