|Overview : |
Terms of Reference
Independent Evaluation Team (Team Leader + Team Member/Expert)
UNDP/UNODC: United Nations Pacific Regional Anti-Corruption Project (UN-PRAC)
COUNTRY OF ASSIGNMENT:
TIMEFRAME OF ASSIGNMENT:
Home-based, with Team leader’s travel to Fiji and possibly 1 other Pacific country
13 August – 19 December 2018
1) GENERAL BACKGROUND
The second phase (2016-2020) of the joint UNDP-UNODC Pacific Regional Anti-Corruption Project (UN-PRAC) aims to support Pacific Island Countries (PICs) to strengthen their national integrity systems. This is in order to promote ‘clean’ governments and to create an enabling environment for trade, business, investment and sustainable development. In turn, this will enhance the delivery of equitable and high-quality services to all Pacific Islanders.
Over the past years, the debate has shifted from ‘why’ countries should be preventing and fighting corruption to ‘how’. The UN Convention against Corruption (UNCAC) is the only international legally binding framework on how to prevent and fight corruption. It provides a solid basis upon which PICs can develop sustainable anti-corruption reforms. It is for this reason that this Project was designed to build on the platform of UNCAC, as well as the efforts undertaken during the first phase (2012-2016) of UN-PRAC. This includes leveraging the recognition by PICs of the UN as a trusted, impartial partner. One medium by which PICs are addressing the ‘how’ question is through the Mechanism for the Implementation of UNCAC (UNCAC Review Mechanism). This requires States parties to consider what national legislative, institutional and practical frameworks are in place to effectively address corruption. Another related anchor is the Development Agenda 2030 and the Sustainable Development Goals (SDGs), recently adopted by UN Member States. SDG 16 (Promote just, peaceful and inclusive societies) directly calls for stronger action on anti-corruption, transparency and accountability. This Project also strives to more coherently address the link between anti-corruption and development, and to integrate anti-corruption into national and regional development processes.
One of the key implementation principles of UN-PRAC, as noted in the project document, is the integration of gender and human rights throughout the Project. Both UNDP and UNODC are committed to mainstreaming gender equality in their programme work and as such, this Project is also committed to gender equality. In addition, where appropriate, specific activities in support of gender equality in the anti-corruption context are considered. The Project reporting is designed to be gender responsive and raise gender-related issues to the extent possible.
The goal of UN-PRAC is to promote and strengthen measures to prevent and fight corruption more efficiently and effectively in the Pacific region. This aligns with the purpose of UNCAC in paragraph 1(1) and the spirit of SDG 16.
In particular, the project aims to:
- Provide Niue, Samoa and Tonga with sufficient information and support to enable their accession to UNCAC and all Pacific States parties to actively participate in the UNCAC review process
- Support Pacific States parties to more effectively implement UNCAC and work towards the achievement of SDG 16
- Strengthen social accountability mechanisms and the anti-corruption role of non-State actors
2) OBJECTIVES OF THE ASSIGNMENT
In line with the project document, the mid-term Independent Project Evaluation will be undertaken around 24 months after the initiation of the project. The aim of the evaluation is to assess the effectiveness of the design and relevance of the project. Its results will be used to inform the implementation of the second half of the project, assess project’s successes and good practices, as well as lesson learnt and areas of improvement. The main users of the evaluation results will be project managers and donors.
The following DAC (Development Assistance Committee) criteria will be assessed during the evaluation: relevance, efficiency, effectiveness, impact and sustainability. In addition, design, established partnerships and cooperation as well as aspects of human rights and gender mainstreaming will be assessed. The evaluation will specifically assess how gender aspects have been mainstreamed into the project. Furthermore, lessons learned and best practices will be identified and recommendations based on the findings formulated.
The outcomes of the evaluation will inform as to what extent the project is contributing to the outcomes of the UNDP and UNODC relevant corporate strategic documents, and above towards the operationalization of the SDG agenda.
3) SCOPE OF WORK
Under the guidance and supervision of the Head of the Integrated Results Management Unit (IRMU) of the UNDP Office in Fiji and the Chief or Deputy Chief of the Independent Evaluation Unit (IEU) of UNODC, the key responsibilities of the evaluation team include (i) development of the evaluation design with detailed methods, tools and techniques, sensitive to key gender as well as human rights issues (ii) ensuring adherence to the United Nations Evaluation Group (UNEG) Norms and Standards, UNODC evaluation norms, standards, guidelines and templates and the full evaluation Terms of Reference (ToR), and (iii) ensuring that all deliverables are submitted in a timely and satisfactory manner and in line with the quality criteria checklist.
The mid-term Independent Project Evaluation (MTE) should assess the design of the project. The following are the key evaluation questions:
The Design of a project or programme measures the extent to which the logical framework approach was adopted.
- To what extent has the results based framework been a useful programme management tool and allowed for an assessment of project outcomes and impact?
- How well aligned are activities, outputs and outcomes in the logical framework?
- To what extend does the design of this project enable optimal use of resources and cooperation with other development initiatives?
- How effective has the joint partnership between UNODC and UNDP been in managing and implementing the project?
Relevance is the extent to which the activity is suited to the priorities and policies of the target group, recipient and donor.
- How relevant is the project to target groups’ needs and priorities, including target groups of governments, development partners and CSOs?
- To what extent are the outputs, outcomes and objectives of this project/programme relevant and contributing to the achievement of the Sustainable Development Goals and how have project activities supported partners in implementing the SDGs?
Efficiency measures the outputs - qualitative and quantitative - in relation to the inputs.
- To what extent were the human and financial resources and inputs converted to outputs in a timely and cost-effective manner?
- To what extent was the coordination between UNODC and UNDP at different levels efficient and appropriate?
- How well has the joint programming model been working?
- What have been its strengths and weaknesses?
- How can such coordination be further strengthened?
Effectiveness is a measure of the extent to which an aid activity attains its objectives.
- To what extent were the planned outputs and outcomes in the project document (and subsequent project revision) achieved?
- Are there any good practices and successes, as well as failures, challenges and areas for improvement?
- Were there any unintended results achieved beyond those included in the logical framework? If so, what were those results?
Impact is the positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended.
- To what extend is UN-PRAC likely to achieve/already achieved its objectives or parts of it beyond the delivery of activities and progress against output targets?
- Which best fit can be identified for adaptation and replication (e.g. in other projects or topics), up-scaling, or prioritization, to ensure achieving outcomes in the most effective way?
Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn.
- To what extent has the ownership of key stakeholders been sought and institutionalized?
- Have the project’s activities contributed to outputs, processes, networks etc. that are likely to have some enduring benefit? What have been the barriers to sustainability?
Partnerships and cooperation
The evaluation assesses the partnerships and cooperation established during the project/ programme as well as their functioning and value.
- To what extent have partnerships been sought and established with and between governments, parliaments, the private sector, civil society and academia?
- What evidence is there of improved capacity of non -State actors to engage in anti-corruption processes?
- To what extent is the project/programme cooperating with other potential partners (including UN agencies, CSOs, academia, etc.) to contribute to the achievement of the SDGs?
Human rights, gender equality, and leaving no one behind
The evaluation needs to assess the mainstreaming throughout the project/programme of human rights, gender equality, and the dignity of individuals, i.e. vulnerable groups.
- To what extent are human rights considerations included in the project design and implementation?
- To what extent has the Project promoted women’s participation throughout the Project activities and improved the active participation of women in discussions and decision-making fora?
- How could gender equality considerations be further included in the project design and implementation?
Lessons learned and best practices
Lessons learned concern the learning experiences and insights that were gained throughout the project/ programme.
- What lessons, both positive and negative, can be learned from this Project?
- What best practices, if any, in planning and implementing UN-PRAC can be identified that should be replicated and/or scaled up in the remainder of UN-PRAC implementation?
Given that this is a Mid-Term Independent Project Evaluation, the emphasis will be on identifying lessons learnt, with a view to adjusting the project design and implementation accordingly. The MTE will therefore make recommendations for the way forward, based on progress thus far.
Findings and lessons learned:
- Outline, as logically and objectively as possible, findings and conclusions
- Highlight the major successes and good practices
- Highlight the major shortcomings, and weaknesses in order of importance
- Present recommendations for a way forward and potential corrective actions; recommendations should be objective, realistic, practical, understandable and forward looking
- Link the recommendations logically to the findings
- Recommend a realistic duration for implementation of remaining project activities
MTE Methodology and Approach
The MTE team should consist of two international consultants (Team leader and Team Member / Expert).
The evaluation will use methodologies and techniques as determined by the specific needs for information, the questions set out in the ToR for the evaluation and the availability of stakeholders. The methodologies and approach need to take into account that this is a regional project, covering 15 countries in a geographically scattered region. In all cases, the evaluation team is expected to analyse all relevant information sources, such as reports, programme documents, thematic programmes, programme files, financial reports and any other documents that may provide further evidence for triangulation, on which his/her conclusions will be based. The evaluation team is also expected to use relevant quantitative and/or qualitative tools as a means to collect relevant data for the evaluation. While maintaining independence, the evaluation will be carried out based on a participatory approach, which seeks the views and assessments of all parties identified as the key stakeholders of the project/ programme, the Core Learning Partners (CLP).
The evaluation ToR provides basic information as regards to the methodology, which should not be understood as exhaustive. It is rather meant to guide the evaluation team in elaborating an effective, efficient, and appropriate evaluation methodology that should be proposed, explained and justified in the Inception Report.
In addition, the evaluation team will be asked to present a summarized methodology (including an evaluation matrix) in the Inception Report outlining the evaluation criteria, indicators, sources of information and methods of data collection. The evaluation methodology must conform to the United Nations Evaluation Group (UNEG) Norms and Standards as well as the UNODC Evaluation Policy, Norms and Standards.
While the evaluation team shall fine-tune the methodology for the evaluation in an Inception Report, a mixed-methods approach of qualitative and quantitative methods is mandatory due to its appropriateness to ensure an inclusive methodology. Special attention shall be paid to an unbiased and objective approach and the triangulation of sources, methods, data, and theories. Indeed, information stemming from secondary sources will be cross-checked and triangulated through data retrieved from primary research methods. Primary data collection methods need to be gender-sensitive as well as inclusive.
The credibility of the data collection and analysis are key to the evaluation. Rival theories and competing explanations must be tested once plausible patterns emerge from triangulating data.
The limitations to the evaluation need to be identified and discussed by the evaluation team in the Inception Report, e.g. data constraints (such as missing baseline and monitoring data). Potential limitations as well as the chosen mitigating measures should be discussed.
When designing the evaluation data collection tools and instruments, the evaluation team needs to consider the analysis of certain relevant or innovative topics in the form of short case studies, analyses, etc. that would benefit the evaluation results.
Bidders shall be free to propose their own team composition and roles. The team should consist of two experts:
- Consultant 1: Team Leader – Evaluation Specialist, with some previous knowledge of anti-corruption, transparency, accountability
- Consultant 2: Team Member/ Expert – Anti-Corruption / Integrity Specialist
Overall responsibilities lie with the MTE Team Leader, who will provide guidance and leadership on conducting the MTE and on preparing/revising the deliverables. The Team Leader will have expertise in results-based evaluation methodologies.
The Team Leader will be responsible for the quality and timely submission of his/her specific deliverables, as specified below, interacting with the UNODC IEU and UNDP IRMU throughout the evaluation process. All products should be well written, inclusive and have a clear analysis process.
Specific responsibilities of the Team Leader include:
- Leading the documentation review and final framing of MTE questions
- Draft inception report, containing: initial observations of the desk review, refined evaluation questions, data collection instruments (including surveys/questionnaires and interview guides), sampling strategy, evaluation matrix and limitations to the evaluation; in line with UNODC evaluation norms, standards, guidelines and templates.
- Leading the MTE team in planning and execution of the assignment
- Managing the division of labor and supervising the work of the other consultant (MTE Team Member)
- Incorporating the use of best practice with respect to M&E and results-based evaluation methodologies
- Leading interviews and consultations, as well as any debriefings to the stakeholders/partners
- Leading the drafting and finalization/quality control of the inception report and the draft and final MTE report, as well as Evaluation Brief (2-pager)
The MTE Team Member/Expert will support the Team Leader and provide the expertise on anti-corruption and integrity in the context of the evaluation. S/he may be asked to lead on specific areas and will be responsible for drafting relevant parts of the inception report and the draft and final MTE report, as well as the Evaluation Brief. Overall responsibilities lie with the MTE Team Leader, who will provide guidance and leadership on conducting the MTE and on preparing/revising the deliverables.
Specific responsibilities of the MTE Team Member include:
- Under the guidance of the Team Lead, undertaking the documentation review and final framing of MTE questions on areas related to anti-corruption
- Supporting the Team Leader during interviews and consultations, as well as any debriefings to the stakeholders/partners, and leading on anti-corruption aspects, as requested by the Team Leader
- Contributing to the drafting and revision of relevant parts of the inception report and the MTE draft and final report, as well as the Evaluation Brief (2-pager), under the guidance of the Team Leader
The evaluators must not have been involved in the design and/or implementation, supervision and coordination of and/or have benefited from the programme/project or theme under evaluation. The evaluation team will report exclusively to the chief or deputy chief of the UNODC IEU and head of the UNDP IRMU, who are the sole clearing entities for all evaluation deliverables and products. The evaluators shall respect the UNEG Ethical Guidelines.
4) DURATION OF ASSIGNMENT, DUTY STATION AND EXPECTED PLACES OF TRAVEL
This assignment will take place between 13 August and 19 December 2018 and is output-based. Travel to Fiji and potentially one other Pacific country is required to hold consultations. Stakeholder from other countries will be interviewed via skype or phone due to the vast geographical area covered by the project and expensive and challenging travel options in the region. Therefore, other work will be home-based. Travel will be facilitated by UN-PRAC as per UNDP’s travel rules and regulations.
- Inception report: by 10 September 2018
- Consultations: by 30 September 2018
- Draft MTE report: by 12 November 2018
- Final MTE report and Evaluation Brief: by 19 December 2018
The MTE is expected to take a total of 67 working days: 40 working days for Consultant 1 (MTE Team Leader) and 27 working days for Consultant 2 (MTE Team Member / Expert), as follows:
Desk review and drafting of Inception Report
13 August – 26 August 2018
(10 working days for lead evaluator and 7 for team member)
Draft Inception Report in line with UNODC evaluation norms and standards
Review of draft Inception Report by IEU and IRMU
27 August – 2 September
Comments on the draft Inception Report to the evaluation team
Incorporation of comments from IEU and IRMU
3 September – 9 September
(2 w/d for lead evaluator and 1 for team member)
Revised draft Inception Report
Deliverable A: Final Inception Report in line with UNODC evaluation norms, standards, guidelines and templates
By 10 September
(overall 12 w/d for lead evaluator and 8 for team member)
Final Inception Report, to be cleared by IEU and IRMU at least one week before the field mission can get started
Evaluation interviews and mission: briefing, interviews with UNODC and UNDP staff (including by phone/skype); observation; focus groups; presentation of preliminary observations (if applicable)
17 September-30 September
(10 w/d for lead evaluator and 8 w/d for team member)
Interviews and data collection
Drafting of the evaluation report
1 October - 14 October
(7 w/d for lead evaluator and 6 for team member)
Draft evaluation report
Review of IEU and IRMU for quality assurance and Project Management for factual errors
Comments on the draft evaluation report to the evaluation team
Consideration of comments from the project manager and incorporation of comments from IEU and IRMU
29 October - 11 November
(4 w/d for lead evaluator and 2 for team member + IEU and IRMU)
Revised draft evaluation report
Deliverable B: Draft Evaluation Report in line with UNODC evaluation norms, standards, guidelines and templates
By 12 November
(overall 21 w/d for lead evaluator and 16 for team member)
Draft evaluation report, to be cleared by IEU and IRMU
IEU to share draft evaluation report with Core Learning Partners for comments, copying IRMU
13 November - 25 November
Comments of CLPs on the draft report
Consideration of comments from Core Learning Partners and preparation of draft Evaluation Brief
(3 w/d for lead evaluator and 1 for team member)
Revised draft evaluation report
Final review by IEU and IRMU; Incorporation of final comments from IEU and IRMU and finalization of report and Evaluation Brief (can entail various rounds of comments from IEU and IRMU)
(3 w/d for lead evaluator and 2 for team member)
Revised draft evaluation report; draft Evaluation Brief
Presentation of evaluation results
Tentative: 19 December
(1 w/d for lead evaluator)
Presentation of evaluation results
Deliverable C: Final evaluation report; presentation of evaluation results; Evaluation Brief (2-pager)
By 19 December
(overall 7 w/d for lead evaluator and 3 for team member)
Final evaluation report; Evaluation Brief and presentation of evaluation results, both to be cleared by IEU and IRMU
5) FINAL PRODUCTS
The final products for this assignment are as follows:
Inception report: The inception report should be prepared by the MTE team before going into the full-fledged MTE exercise. It should include initial observations of the desk review, refined evaluation questions, data collection instruments (including surveys/questionnaires and interview guides), sampling strategy, evaluation matrix and limitations to the evaluation, in line with UNODC evaluation norms, standards, guidelines and templates.
Draft MTE report: Draft evaluation report should be prepared in line with UNODC evaluation norms, standards, guidelines and templates, including an analysis of the performance of the project to adequately address gender equality as well as human rights issues, with concrete findings, conclusions and recommendations.
MTE report: The final report will be produced by the MTR team based on feedback received on the draft report. The final report will be shared with stakeholders and other partners. The final evaluation report and an Evaluation Brief (2-pager) should be prepared in line with UNODC evaluation norms, standards, guidelines and templates.
6) PROVISION OF MONITORING AND PROGRESS CONTROLS
The evaluation team will work under the supervision of the UNODC IEU and UNDP IRMU, who will consult with and seek inputs from other counterparts in line with the evaluation ToR.
UN-PRAC will support the logistical arrangements of consultant travels and stakeholder consultations. Although UN-PRAC is administratively responsible for the MTE, it shall not interfere with analysis and reporting, except where requested and at opportunities for comments/feedback.
UNODC and UNDP will share the final version of the MTE report with the national stakeholder agencies and all partners of the project.
7) TEAM COMPOSITION, DEGREE OF EXPERTISE AND QUALIFICATIONS
The MTE team will be composed of two independent consultants not involved with the formulation, appraisal, approval and/or implementation of daily management of the project. Both UNODC and UNDP will have the opportunity to weigh in on the selection of the consultants and provide inputs to the required qualifications and competencies. UNDP will oversee the administrative process of recruitment and contracting, with both UNODC and UNDP being given the option of participating in each stage of the selection, including short-listing, reviewing financial proposals, interview panels, etc.
The following are required qualifications for MTE Team Lead:
Education: Master’s degree in Law, Development Studies, Public Finance, Political Science, Social Sciences, Evaluation, or other relevant field and preferably formal training/educa