Procurement Notices

INDEPENDENT TERMINAL EVALUATOR OF THE PROJECT CLIMATE TWIN PHOENIX - RESILIENCE AND PREPAREDNESS TOWARD INCLUSIVE DEVELOPMENT (PCTP-RAPID) PROGRAM

(Individual Contractor- National)

Background and context

The Project Climate Twin Phoenix – Resilience and Preparedness toward Inclusive Development (PCTP-RAPID) Program is a capacity development program supporting the long-term recovery of identified Local Government Units (LGUs) and communities in Sendong (Washi), Pablo (Bopha), and Yolanda (Haiyan)-affected areas in the cities of Cagayan De Oro and Iligan in Region 10, provinces of Davao Oriental and Compostela Valley in Region 11, and provinces of Leyte and Samar in Region 8 by having their disaster and climate risks mitigated and mainstreamed into land use and development planning, programming, regulatory, and other implementation processes.

It aims to address the institutional capacity and individual competency gaps of key players on disaster risk reduction and management of natural hazards including those aggravated by climate change. It also aims to raise the awareness and competencies of decision makers and communities in the target areas about the impacts of natural hazards on lives, properties and the economy, and that the changing climate brings extreme weather events that can trigger and exacerbate the impacts of future hazard events.

The activities of PCTP-RAPID are designed to enable the target LGUs to come up with better plans, policies, and regulatory measures that consider major challenges that affect the wellbeing of the population, including climate and disaster risks. The entirety of the program has 7 outputs:

1. Climate/disaster risk and vulnerability assessments produced as a basis for “climate/disaster proofing” future development in the target areas.
2. Priority disaster mitigating measures such as community-based and managed early warning systems (CBMEWS), contingency plans, re-engineering standards, and other resilience building interventions developed and implemented.
3. Competencies of local governments and critical partners improved to deal with the disaster risks of multi-hazards, including those from climate change and general level of awareness and competencies of vulnerable communities and other local stakeholders increased to deal with disaster and climate change risks.
4. Disaster Risk Reduction (DRR) and Climate Change Adaptation (CCA) mainstreamed into land use, socio-economic plans, and investment programs at the national and local level.
5. Risk sharing/transfer mechanisms developed and showcased.
6. Knowledge management on Disaster Risk Reduction and Management and Climate Change Adaptation developed and implemented.
7. Competencies of barangays and communities improved to deal with the disaster risks of multi-hazards, including those from climate change; and general level of awareness and competencies of vulnerable communities and other local stakeholders increased to deal with disaster and climate change risks.

RAPID covers 12 Yolanda (Haiyan)-affected municipalities located along the coastline of the Leyte Gulf: Tacloban City, Palo, Tanauan, Tolosa, Mayorga, MacArthur, Dulag, and Abuyog in Leyte, Basey and Marabut in Western Samar, and Lawaan and Balangiga in Eastern Samar. It is an expansion of PCTP that was executed in response to the impacts of Sendong (Washi) and Pablo (Bopha) which devastated many areas in Regions 10 and 11, particularly the cities of Cagayan de Oro and Iligan, and Compostela Valley and Davao Oriental, respectively. PCTP commenced in 2012 and was completed in 2015, while RAPID started in 2014 and was expected to be completed by the end of 2017. However, it was extended for another year to be completed in 2018. A total of A$9.3 million in funding was provided for the whole of PCTP-RAPID by the Australian Government through the Australian Embassy in the Philippines – Department of Foreign Affairs and Trade (DFAT). It is administered and implemented by the United Nations Development Programme (UNDP) Philippine Country Office through its Project Management Unit (PMU). From 2012 to 2017, PCTP-RAPID was implemented by the Climate Change Commission (CCC) through UNDP’s National Implementation Modality (DIM), and for 2018 is implemented by UNDP under a Direct Implementation Modality (DIM).

In 2015, UNDP commissioned an independent midterm evaluation of PCTP-RAPID. For 2018, an independent terminal evaluation shall be conducted.

Evaluation purpose, scope, and objectives

The terminal evaluation shall be conducted in relation to the operational closure of the RAPID program by the end of 2018. It finds its basis in Article VI of the Cost Sharing Agreement between UNDP and Australian Aid for International Development (AusAID) and is also a mandatory activity for all UNDP-implemented projects and programs. As such, this evaluation aims to assess RAPID’s value additions and how its results can be adopted and sustained by national and subnational government entities, communities, civil society groups, and other stakeholders. The evaluation will do this by identifying the relevant information and approaches that may be picked up on to inform and enhance policies, programming, and practices on resilience building towards a more effective development agenda. The findings will be related to national and international development priorities and objectives (e.g. Philippine Development Plan, Australia’s Aid Investment Plan for the Philippines), international frameworks (e.g. Sustainable Development Goals), and internal programming and planning documents (e.g. UNDP Country Programme Document, Strategic Plan) [OA7] to draw specific lessons, conclusions, and recommendations for future similar interventions. It will also cover PCTP to the extent that the questions and recommendations from the midterm evaluation are reviewed and answered.

The terminal evaluation will assess the actual performance of PCTP-RAPID and its contributions against its targets and expected results from 2012 to 2018. Gender equality is a significant objective of the investment and a key priority for UNDP and the Australian and Philippine governments. As such the review will also assess PCTP-RAPID’s gender equality results. The evaluation will involve all of the target areas from Regions 8, 10, and 11, and the following stakeholders:

1. Provincial, City, Municipal, and Barangay LGUs and departments/offices/committees including, but not limited to:
   1. Disaster Risk Reduction and Management Offices
   2. Planning and Development Offices
2. National and regional government agencies:
   1. Climate Change Commission
   2. Department of Interior and Local Government
   3. National Economic and Development Authority
   4. Philippine Atmospheric, Geophysical, Astronomical Services Administration
   5. Office of Civil Defense
   6. Housing and Land Use Regulatory Board
3. Project Management Unit
4. United Nations Development Programme
5. Australian Embassy – Department of Foreign Affairs and Trade

The evaluation must consider whether capacities, processes, and products related to DRR and CCA have been built/installed or enhanced/mainstreamed (climate and disaster risk assessment, plan preparation, investment programming, among others) and if these have been introduced. It must answer if the above-mentioned expected results are achieved, or if the intervention created conditions for achieving them; whether gaps are present, which need remediation, and where further interventions will yield further returns.

An assessment should be made about scaling results for national adoption and identify conditions for sustainability. It must also consider whether resources have been properly and judiciously harnessed towards implementation and delivery of stated outputs and the extent to which these outputs contributed to observed results achieved. The evaluation must also identify any operational issues that may be improved to facilitate better program implementation and delivery for similar programs in the future  

Specifically, the Consultant(s) is/are expected to undertake the following tasks:

1. Assess PCTP-RAPID’s performance relative to its objectives and targets as stated in reference documents including, but not limited to:

• Project Document
• Theory of Change/Logical Framework/Results Framework
• Work Plans
• Monitoring & Evaluation Plans
• Progress Reports
• Evaluation Reports;

      2. Assess the appropriateness of the program’s overall/per component implementation framework, methodologies, and strategies in       

         achieving the set objectives, outputs, and results as well as in putting in place models or practices which the government, communities, and other stakeholders could adopt;

      3. Assess the effectiveness and efficiency in the use of program resources to meet target outputs and results, taking into consideration the principle of value for money;

      4. Analyze factors including the project management/operational setup and its degree of influence in the achievement or non-achievement of target outputs and results;

      5. Assess the relevance and effectiveness of the program’s partnerships and other implementation strategies and highlight which among these methodologies and approaches could be sustained or replicated by government agencies, communities, and other stakeholders;

      6. Determine capacities, processes, and products developed and the level of participation/degree of ownership of stakeholders in the achievement of the outputs and results;

      7. Document and draw up lessons learned, good, replicable and/or innovative practices, cross-cutting issues (e.g. gender mainstreaming, human rights, DRRM, resiliency building, beneficiary selection, stakeholder participation, etc.) and recommendations on appropriate project strategies to improve future programming on resilience building;

      8. Assess RAPIDs performance with regards to the integration of gender equality and the achievement of gender equality results;

      9. Put forward some policy and program recommendations to UNDP as direct implementer of the project; and

      10. Make recommendations to DFAT to support future programming.

Evaluation criteria and key guiding questions (Complete list of guide questions are in the ATTACHED TERMS OF REFERENCE)

The terminal evaluation will follow the Organization for Economic Cooperation and Development – Development Assistance Committee (OECD-DAC) criteria, and the questions are grouped according to relevance, effectiveness, efficiency, and sustainability, and impact. It will also have a focus on gender equality. Note that the questions are not exhaustive and may change, depending on a consultative determination with the Evaluation Reference Group. The set of evaluation questions shall be finalized through the inception report.

Methodology

The overall approach and methodology of the terminal evaluation shall be guided by the provisions set forth in the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results and the UNEG Norms and Standards for Evaluation. It should be conducted in accordance with the principles outlined in the UNEG Ethical Guidelines for Evaluation. The Consultant will be required to sign an Evaluation Consultant Code of Conduct Agreement Form.

The Consultant is expected to frame the evaluation effort using the OECD-DAC universal criteria of relevance, effectiveness, efficiency, and sustainability, impact, as well as gender equality. A rating scale for each criterion and overall program performance will have to be defined by the Consultant and must include a description for each rating as basis for interpretation. The list of key evaluation questions and sub-questions, which shall form part of technical proposal of the Consultant, should draw out the required information for each evaluation objective and be classified according to the criteria they belong to. The list will have to be finalized with the Project Management Unit and shall be included in the Consultant’s Inception Report.

The evaluation should employ both qualitative and quantitative evaluation methods and instruments, where applicable. The technical proposal of the Consultant must indicate specific activities, data sources, and data collection and analysis methods needed to meet the evaluation purpose and objectives. These may include, but are not limited to: desk review of project documents, on-site validation of tangible outputs, surveys, key informant interviews, and focus group discussions. The Consultant is expected to follow a participatory and consultative approach with the PMU, responsible partners, direct beneficiaries, and other program stakeholders.

Evaluation products (deliverables)

The Consultant is expected to come up with the following products for the terminal evaluation:

INCEPTION REPORT: The inception report should generally illustrate and explain the overall design and method of the terminal evaluation and reflect any agreed recommendations arising from the Inception Meeting. It should include an evaluation matrix, which identifies the key evaluation questions and how they will be answered by the methods selected. The inception report should be produced before any formal evaluation activities can begin.

The inception report must be submitted within 7 working days after the Inception Meeting.

EVALUATION DEBRIEFING: A debriefing shall be requested immediately following the conclusion of evaluation activities, schedules permitting. A presentation on the actual coverage of the evaluation, preliminary findings, and next steps will be sought from the Consultant. Any additional requirements that may be deemed necessary by UNDP or the PMU will be discussed during the debriefing and will have to be incorporated in the evaluation reports.

DRAFT EVALUATION REPORT WITH PRESENTATION: The evaluation report with the corresponding data and findings will be presented in the draft stage to key stakeholders for review and revision. Any comments on the draft shall be addressed in the final report. The draft evaluation report must, at the minimum, follow the Evaluation Report Template as outlined in Annex 7 of the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results. It shall be subjected to the quality checking and assurance criteria outlined in the UNEG Quality Checklist for Evaluation Reports. A presentation shall accompany the report, and both products shall be submitted within 7 working days after the evaluation debriefing.

FINAL EVALUATION REPORT WITH PRESENTATION: Following the comments in the draft stage, the Consultant must prepare a final evaluation report. An evaluation report audit trail must be prepared, detailing the comments raised in the review and the changes/responses made by the Consultant to show how the comments have been addressed. Likewise, the final evaluation report must, at the minimum, follow the Evaluation Report Template as outlined in Annex 7 of the UNDP Handbook on Planning, Monitoring and Evaluating for Development results. It shall be subjected to the quality checking and assurance criteria outlined in the UNEG Quality Checklist for Evaluation Reports. A presentation shall accompany the report, and both products shall be submitted within 7 working days after the submission of the draft evaluation report.

The Consultant is also expected to turn over to UNDP all materials related to the evaluation (e.g. raw and processed data, pictures, list of respondents and written/signed consent forms, etc.).

Evaluator’s required competencies

UNDP seeks to engage the services of an Independent Evaluation Consultant to carry out the Independent Terminal Evaluation of PCTP-RAPID. The Consultant will have the overall responsibility during all phases of the evaluation, particularly in ensuring the high quality and timely completion of evaluation processes, methodologies, and outputs. In close collaboration with the PMU and UNDP, he/she will lead the implementation of the evaluation design, guide the methodology and application of data collection instruments, and lead the consultations with stakeholders. At the reporting phase, he/she is responsible for putting together the first comprehensive draft and the final version of the evaluation report, based on inputs from the PMU, UNDP, and stakeholders. The applicant should possess the following qualifications and must obtain a passing score of at least 70 points:

The applicant should not have been involved, in any way or the other, in designing, executing, or advising in any component or activity of PCTP-RAPID that is covered by the evaluation. Failure to declare this information prior to the award of contract may be considered as ground for cancellation of the engagement.

Implementation arrangements

The principal responsibility for managing this evaluation resides with the PMU through the Project Manager (PM) and the Monitoring and Evaluation Officer (MEO). Both will be responsible for liaising with the Evaluation Team pertaining to required technical and financial documents, coordinating with stakeholders, setting up interviews, arranging field visits, and looking after the evaluation budget and schedule. They shall likewise assist in distribution of draft reports to stakeholders for their review, consolidation of comments, and in organizing key stakeholders’ meetings for presentation of the salient points of the draft/final reports.

The Inclusive and Sustainable Development Unit (ISDU) and Management Support Unit (MSU) will provide support in the procurement process for the selection of a service provider (i.e., publication of the TOR and assessment of proposals), briefing the Evaluation Team on UNDP evaluation norms and standards, reviewing and quality assuring the inception/draft/final reports, and in publishing findings and management responses at the UNDP Evaluation Resource Center.

The Consultant will be responsible for implementing all evaluation-related activities and in producing the evaluation products listed in the deliverables section of this TOR. While the PMU will provide the information required and support in coordinating with stakeholders, the Evaluation Team will have to manage its own schedule and logistical arrangements in the conduct of interviews and site visits. 

The selected Consultant shall be remunerated based on the following payment schedule:

Timeframe for the evaluation process

The whole evaluation process is expected to commence in 2019, with all related activities expected to be completed by the end of project activities, following the table below:

Application submission process and criteria for selection

Submission of proposals is open to all interested and qualified individuals. The proposal shall contain both technical and financial components and should be submitted to the email address indicated in the Procurement Notice. The proposal shall include the following:

1. Duly accomplished Letter of Confirmation of Interest and Availability using the template provided by UNDP (see attached);
2. Personal CV or P11 form (see attached P11 UNDP Personal History Form), indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the applicant and at least three (3) professional references;
3. At least 2 final/published versions of terminal evaluation reports to provide the UNDP Proposal Assessment Team with an idea on how the applicant packages reports (i.e. quantitative and qualitative presentation of findings and recommendations);
4. Plan of Approach and Methodology- The Plan of Approach and Methodology shall contain the same elements of the inception report as indicated in Section E; and
5. Financial Proposal indicating the all-inclusive fixed total contract price, supported by a breakdown of costs, using the template provided by UNDP (see attached template).

The technical proposal shall be evaluated based on the following criteria:

1. Qualifications (Background/experience of Consultant and 2 sample outputs)            30%
2. Plan of Approach and Methodology                                                                                  70%
   1. Approach to answering evaluation questions -35%
   2. Appropriateness of methods and instruments-35%

The financial proposal should indicate both the breakdown and total costs of the engagement. It should be all-inclusive covering professional fees, travel expenses (i.e. vehicle rental and airfare), office supplies, food and accommodation, overhead costs, insurance coverage, and other incidental expenses. 

The proposal shall be evaluated using the Combined Scoring Method, where the qualifications and methodology will be weighted a maximum of 70% and combined with the price offer which will be weighted a maximum of 30%.

Annexes

UNDP Handbook on Planning, Monitoring and Evaluating for Development Results

UNEG Norms and Standards for Evaluation

UNEG Ethical Guidelines for Evaluation

UNEG Quality Checklist for Evaluation Reports

Please email applications to procurement.ph@undp.org by 5 December 2018. Please make sure emails do not exceed 4MB, and that the abovementioned requirements.

Kindly also refer to the attachments to this Procurement Notice:

1) Terms of Reference for complete details on the vacancy

2) 2018 Letter of Confirmation of Interest and Availability and Financial Proposal (template)

3) P11 Personal History Form (template)

4) UNDP General Conditions for Individual Contractors