Procurement Notices

The interested offeror must submit the application via the following link

https://jobs.undp.org/cj_view_job.cfm?cur_job_id=100704

Umbrella Programme for Socio-Economic Development

Mid-Term Evaluation TOR

1. Background and context

The Umbrella Programme for Socio-Economic Development Project, launched in May 2019, is designed to support mainstreaming social and economic development in the national policies in Saudi Arabia. The ultimate programme objective is to strengthen evidence-based policy planning and decision-making to support the realization of Saudi Vision 2030 and the Sustainable Development Goals (SDGs).

The review of the first year of the project implementation identified a need for additional programme efforts to enable the project achieves its objectives. As a result, UNDP and MOEP have agreed to revise the Umbrella Programme for Socio-Economic Development Project to increase policy engagement, reinforce capacity development and strengthen monitoring and evaluation. The revised project will enable more coherent and structured support to policy-making, policy planning and assessment. The amendment takes into account the needs for technical support of the various deputyships in MOEP as well as the support needed to strengthen national and sub-national partners’ engagement in socio-economic policy-making. The amended  programme framework  will cover the following main areas:

•             Strengthening MOEP capacity for policy-making, policy analysis and evaluation

•             SDGs monitoring and reporting

•             Support to regional development planning

•             Producing the National Human Development Report

•             Assessing the impacts of the COVID-19 on the Saudi economy

Basic project information can also be included in table format as follows:

1. Evaluation purpose, scope and objectives

This is a mandatory mid-term evaluation planned half-way through the life cycle of the project and triggered by a substantial budget revision with an increased budget and new components in terms of outputs of the project.  As such, it was deemed crucial to undertake this evaluation to ensure the project is on track to meet its targets and deliver the services expected with the impacts its due to demonstrate.

The project has been ongoing since May 2019 and has, thus far, never been evaluated.  Drastic changes have been taking place in the country and the project has had to adapt to the changes over recent years, this included changes in Ministers, Deputy Ministers and Project staff, resulting in changing project directions.  To ensure the project has delivered its intended objectives thus far and to provide recommendations for the way forward, it is imperative to conduct a mid-term evaluation and ensure the project delivery is on track. This evaluation will benefit the Ministry of Economy and Planning in their planning for future years to meet Saudi Vision 2030 and highlight the impacts this project has had on the economy sector over the past few years.

This evaluation will measure the performance in the past two years of the project in terms of delivering on:

1. Strengthening MOEP capacity for policy-making, policy analysis and evaluation
2. SDGs monitoring and reporting
3. Support to regional development planning
4. Producing the National Human Development Report
5. Assessing the impacts of the COVID-19 on the Saudi economy

The evaluation will also address the primary issues of concern to the national partners, namely MEOP and provide recommendations as to the best course of action that needs to be taken to ensure the achievements of intended results.

This evaluation will also provide recommendations as to how the project will, in the future, mainstream gender in development efforts, and, wherever possible, also consider disability issues and the rights-based approach

1. Evaluation criteria and key guiding questions

Evaluation questions define the information that the evaluation will generate. To assess the impact of this project in the grand scheme of Outcomes, the following are guiding and crucial, but in no way exclusive, questions the evaluation must answer to provide the information needed in order to make decisions, take actions or increase knowledge.

Questions are grouped according to the four or five OECD-DAC evaluation criteria: (a) relevance; (b) coherence; (c) effectiveness; (d) efficiency; and (e) sustainability.  

1. Methodology

This TOR suggests an overall approach and method for conducting the evaluation, as well as data sources and tools that will likely yield the most reliable and valid answers to the evaluation questions within the limits of resources. However, final decisions about the specific design and methods for the evaluation should emerge from consultations with the programme unit, the evaluator and key stakeholders about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, given limitations of budget, time and data. This will have to be reflected in the inception report prior to starting the evaluation mission.

Evaluation should employ a combination of qualitative and quantitative evaluation methods and instruments. The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and male and female direct beneficiaries.  Suggested methodological tools and approaches may include:

• Document review. This would include a review of all relevant documentation, inter alia
  • Project document (contribution agreement).
  • Theory of change and results framework.
  • Programme and project quality assurance reports.
  • Annual workplans.
  • Activity designs.
  • Consolidated quarterly and annual reports.
  • Results-oriented monitoring report.
  • Highlights of project board meetings. 
  • Technical/financial monitoring reports.
• Interviews and meetings with key stakeholders (men and women) such as key government counterparts, donor community members, representatives of key civil society organizations, United Nations country team (UNCT) members and implementing partners:
  • Semi-structured interviews, based on questions designed for different stakeholders based on evaluation questions around relevance, coherence, effectiveness, efficiency, and sustainability.
  • Key informant and focus group discussions with men and women, beneficiaries and stakeholders.
  • All interviews with men and women should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
• Surveys and questionnaires including male and female participants in development programmes, UNCT members and/or surveys and questionnaires to other stakeholders at strategic and programmatic levels.
• Field visits and on-site validation of key tangible outputs and interventions.
• Other methods such as outcome mapping, observational visits, group discussions, etc.
• Data review and analysis of monitoring and other data sources and methods. To ensure maximum validity, reliability of data (quality) and promote use, the evaluation team will ensure triangulation of the various data sources.
• Gender and human rights lens. All evaluation products need to address gender, disability, and human right issues.

The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between UNDP, key stakeholders and the evaluators.

1. Evaluation products (deliverables)

The TOR should clearly outline the outputs UNDP expects from the evaluation team, with a detailed timeline and schedule for completion of the evaluation products. Where relevant, the TOR should also detail the length of specific products (number of pages). These products include:

• Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
• Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
• Draft evaluation report (within an agreed length). A length of 40 to 60 pages including executive summary is suggested.
• Evaluation report audit trail. The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period of time, as outlined in these guidelines. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
• Final evaluation report.
• Presentations to stakeholders and/ or evaluation reference group (if required).
• Evaluation brief and other knowledge products or participation in knowledge-sharing events, if relevant to maximise use.

1. Evaluation team composition and required competencies

This section details the specific skills, competencies and characteristics required of the evaluator / individual evaluators in the evaluation team, and the expected structure and composition of the evaluation team, including roles and responsibilities of team members. This may include:

• Required qualifications: Advanced degree in relevant field, a minimum of ten years’ experience in conducting/ managing evaluations, relevant knowledge, and specific country/regional experience.
• Technical competencies: team leadership skills and experience, technical knowledge in UNDP thematic areas, with specifics depending on the focus of the evaluation, data analysis and report writing etc.
• Technical knowledge and experience: Gender expertise/competencies in the evaluation a strong plus. some knowledge and/or experience of disability inclusion. Technical knowledge and experience in other cross-cutting areas such equality, disability issues, rights-based approach, and capacity development.
• Language skills required: Fluent English speaking, writing, and reading skills, Arabic is an asset

The provision of evidence will be expected to support claims of knowledge, skills and experience to the above in terms of:

• resumes,
• work samples,
• references

The Evaluator is to clearly state independence from any organizations that have been involved in designing, executing, or advising any aspect of the intervention that is the subject of the evaluation.[3] 

1. Evaluation ethics

Explicit statement that evaluations in UNDP will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’.[4]

“This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees, and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.”

1. Implementation arrangements

In order to clarify expectations, eliminate ambiguities, and facilitate an efficient and effective evaluation process, the following is the organization and management structure for the evaluation as well as the definition of the roles, key responsibilities and lines of authority of all parties involved in the evaluation process.

1. The evaluation commissioner: Accountable for the quality and approval of final terms of reference (TORs), final evaluation reports and management responses before final submission to the ERC. This is traditionally the head of office, in the case the Resident Representative of UNDP KSA Country Office.
2. The evaluation manager: Lead the evaluation process and participate in all of its stages - evaluability assessment, preparation, implementation, management and use. In this case it is the M&E Officer
3. The programme manager: Support the establishment of the evaluation reference group with key project partners where needed and participate in calls/meetings on request. Provide inputs/ advice to the evaluation manager on the detail and scope of the TOR for the evaluation and how the findings will be used.  Provide the evaluation manager with all required data (e.g. relevant monitoring data) and documentation (reports, minutes, reviews, studies, etc.), contacts/ stakeholder list etc.
4. The evaluation partner: Participate in the review of key evaluation deliverables, including the TOR, inception report, and successive versions of the draft evaluation report.  Ensure that data and documentation in general, but in particular relating to gender equality and women’s empowerment and other cross-cutting issues, are made available to the evaluation manager
5. The independent evaluator: Fulfil the contractual arrangements under the TOR.  Develop the evaluation inception report, including an evaluation matrix and a gender responsive methodology, in line with the TOR, UNEG norms and standards and ethical guidelines. Conduct data collection and field visits according to the TOR and inception report.  Produce draft reports adhering to UNDP evaluation templates, and brief the evaluation manager, programme/ project managers and stakeholders on the progress and key findings and recommendations.  Consider gender equality and women’s empowerment and other cross-cutting issues, check if all and respective evaluation questions are answered, and relevant data, disaggregated by sex, is presented, analysed and interpreted.  Finalize the evaluation report, incorporating comments and questions from the feedback/audit trail. Record own feedback in the audit trail

1. Evaluation products (deliverables)

The consultant will be expected to deliver the following:

1. Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
2. Evaluation findings debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
3. Draft evaluation report (60 pages including executive summary). The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide a set of comments to the evaluator within an agreed period of time, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in the evaluation guidelines.
4. Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how he/she has addressed comments.
5. Final evaluation report:
   1. Executive summary;
   2. Introduction, including description of the work conducted;
   3. Findings and conclusions; 
   4. Recommendations, including, as applicable, a revised work plan to address the pending tasks and eventual corrective action as well as an improved system for measuring the impact of the project in terms of achieved energy savings;  
   5. Annexes providing a brief summary of the documents reviewed and persons interviewed with the description of the key content / conclusions drawn and any other relevant materials.
6. Validation workshop for presentations to stakeholders
7. Evaluation brief and other knowledge products or participation in knowledge-sharing events, if relevant.
8. The consultant should present three hard copies of the report as well as an electronic copy. The draft final report should be submitted no later than three weeks after the end of the on-site mission and the final report within two weeks from receiving the comments of the project management and UNDP on the draft reports

Standard templates that need to be followed are provided in the Annexes section. It is expected that the evaluator will follow the UNDP evaluation guidelines and UNEG quality check list and ensure all the quality criteria are met in the evaluation report.

In line with UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactory completed due to impact of COVID-19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her/their control.

1. Time frame for the evaluation process

Working day allocation and schedule