Procurement Notices

Institutional Support to Statistics

Terms of Reference

Mid-Term Evaluation

December 2021

1. Background and context

The General Authority for Statistics GASTAT is a governmental entity that enjoys an independent legal personality.

GASTAT is the only official statistical reference for statistical data and information in Saudi Arabia. It executes all the statistical work, in addition to the technical oversight of the statistical sector, which includes a multiple system of centers and statistical units established within the administrative structures of government agencies and some private sector institutions as well.

GASTAT supervises the implementation of the national strategy for statistical work in coordination with the relevant entities. It also designs and implements field surveys, conducts statistical studies and researches, analyzes data and information, in addition to the documentation and archiving works of information and statistical data

that cover all aspects of life in Saudi Arabia from its multiple sources. Data are written, classified, and analyzed; their indicators are extracted as well.

GASTAT prepares statistical classifications and guides according to international standards to be used and updated. It also prepares statistical bulletins and reports of field surveys as well as statistical researches for publishing purposes.

Moreover, it also supervises the formation of a comprehensive system of national databases ofthe various statistical fields available at the data sources. It also works to spread knowledge and statistical awareness, and provides recommendations to public entities to develop all the information systems and methodologies of the statistical work.

The General Authority for Statistics is the official body responsible for providing public entities, private establishments, individuals and international organizations with official statistics in accordance with statutory procedures; in order to support policy and decision-makers in these entities to move forward in implementing the developmental plans. On the other hand, statistical censuses such as the general population, housing and establishments’ census are considered the most important products of GASTAT, in addition to the periodic statistical indicators and indices in the economic, social, health, educational, agricultural, and workforce fields.

GASTAT also provides its services and statistical products through a huge digital base for data, indicators and statistics of various development fields in the Kingdom in different forms and multimedia. That is to facilitate the obtaining of the statistical data and information by the clients in a quick and efficient way, and at the highest level of transparency, accuracy, and integration between the components of the statistical sector.[1]

Statistics and information sector in Saudi Arabia has also received a plenty of demands due to its important position and main role internationally and regionally.

Nationally, the continuous progress in the developmental path in Saudi Arabia also requires more accurate, comprehensive, timeless statistics to support decision and policy making, monitor the progress, and evaluate the impact and performance.

The analysis of the current situation of the statistics sector in Saudi Arabia, which is the base and starting point of preparing the strategy, has showed that this sector is an important priority to the country. The sector has also witnessed a remarkable progress in the statistical production- whether in administrative records or field work for census, surveys, and researches- in addition to the progress in the statistical systems, dissemination methods, and using the latest technologies. However, there are some obstacles and challenges that may face the statistical work and need to be solved in order to achieve a notable improvement in record time, these challenges include: lack of statistical awareness, numbers analysis by non-specialists, not referring to the main sources in some cases, lack of some information in administrative records, lack of providing the required data by some international groups and organization such as: (G20) where Saudi Arabia is a member, as well as inability in drafting and publishing statistical report,, etc.[2]

Since its launch in 2017, the Institutional Support to Statistics project continues to provide an effective cooperation framework for strengthening the statistical capacities in support of Saudi Arabia's Vision for 2030.

The project organised several consultations to understand the technical needs of the statistical system. The project also reviewed its success elements and the drawbacks to improve cooperation and achieve better impact.

The feedback and the needs of the statistical system convinced GASTAT and UNDP to amend their cooperation to further strengthen the statistical system in Saudi Arabia through a substantive revision of the Institutional Support to Statistics project. The revised project provides a framework to deliver demand-driven technical support and timely capacity development interventions aligned with the National Statistical Development Strategy. It also addresses the statistical needs triggered by the context of COVID-19.   Specifically, the technical support will be delivered under the following outputs:

Output 1: GASTAT capabilities to produce statistical data and information in light of the Kingdom's 2030 statistical vision strengthened.

Output 2: GASTAT capacities to integrate statistical analysis into decision-making processes enhanced with focus on achieving the SDGs and Saudi Arabia's Vision for 2030.

Output 3:   Modernisation of the statistical systems enabled

Output 4: Capacities for statistics governance and quality control strengthened

Output 5: Nowcasting solution developed and preliminary early estimates of GDP and economic activity in KSA produced

The project extension period (January 2021 to December 2024) provides the necessary time for the project to complete ongoing activities and deliver the revised outputs to achieve a more significant impact.  

The project document will be provided amongst documents to be shared with the evaluator

Basic Project information is included in the table below:

1. Evaluation purpose, scope, and objectives

The project has been ongoing since February 2016 and has, thus far, never been evaluated.  Drastic changes have been taking place in the country and the project has had to adapt to the changes over recent years, this included changes in Ministers, Deputy Ministers and Project staff, resulting in changing project directions.  To ensure the project has delivered its intended objectives thus far and to provide recommendations for the way forward, it is imperative to conduct a mid-term evaluation and ensure the project delivery is on track. This evaluation will benefit the General Statistics Authority in their planning for future years to meet Saudi Vision 2030 and highlight the impacts this project has had on Statistics sector over the past few years.

This evaluation should cover the project period between (2017 & 2021) and all the components of the project, and their impact on the sector in the whole country. However, the evaluator shall consider the Saudi Vision 2030 objective’s and other GASTAT- Initiatives in formulating the report which can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP Goals.

Especially regarding the intended Outputs:

• How has GASTAT capabilities to produce statistical data and information been strengthened
• Provide proof to how GASTAT capacities were enhanced to integrate statistical analysis into decision-making processes with focus on achieving the SDGs and Saudi Arabia's Vision for 2030
• How was the statistical system modernized? How has data related to social inclusion been integrated?
• Report on how of statistics governance and quality control have been strengthened
• Solutions developed and estimates of GDP forecasted

1. Evaluation criteria and key guiding questions

Evaluation questions define the information that this evaluation will generate. Questions should be grouped according to the four OECD-DAC evaluation criteria: (a) relevance; (b) effectiveness; (c) efficiency; (d) impact and (e) sustainability (and/or other criteria used).  

The mainstream definitions of the OECD-DAC criteria are neutral in terms of human rights and gender dimensions and these dimensions need to be added into the evaluation criteria chosen (see page 77, table 10 of Integrating Human Rights and Gender Equality in Evaluations).

Ratings must be provided on the following performance criteria. The completed table must be included in the evaluation executive summary.

Guiding evaluation questions can be further refined by the evaluator and agreed with UNDP and the- evaluation stakeholders in the inception report.

Evaluation Approach and Methodology

The methodology of work will consist of desk review of relevant project documentation and direct consultations with the project management, staff and other key local stakeholders during 5 days site visit to Riyadh in November 2021.

The overall duration of the assignment is expected to consist of a site visit of 5 days includes a corresponding amount of desk work to pre-review the required project documentation and to finalize the reporting. This makes the total working days to be 20 working days, the timeframe detailed shall be as stipulated below.

In carrying out the evaluation task, the consultant will pay particular attention to the following:

• Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments;
• Document review of all relevant documentation. This would include a review of inter alia;

• Project document (contribution agreement);
• Theory of change and results framework;
• Programme and project quality assurance reports;
• Annual workplans;
• Activity designs;
• Consolidated quarterly and annual reports;
• Results-oriented monitoring report;
• Highlights of project board meetings; 
• Technical/financial monitoring reports;
• GASTAT National Strategy
• Vision 2030 and corresponding transformation plans related to the statistics sector (National Transformation Program - Vision 2030)

• Semi-structured interviews with key stakeholders (men and women) including key government counterparts, donor community members, representatives of key civil society organizations, UNCT members and implementing partners: All interviews should be undertaken in full confidence and anonymity under the support of the Project Management. The mid-term evaluation report should not assign specific comments to individuals
• Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed.
• Surveys and questionnaires including male and female participants in development programmes, UNCT members and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels.
• Field visits and on-site validation of key tangible outputs and interventions.
• The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries including industry partners and general public on awareness.
• Other methods such as outcome mapping, observational visits, group discussions, etc.
• Data review and analysis of monitoring and other data sources and methods.
• Ensure maximum validity, reliability of data (quality) and promote use; the evaluator will ensure triangulation of the various data sources.
• Gender and human rights lens. All evaluation products need to address gender, disability, and human right issues

The evaluator must use gender-based methodology and tools and ensure that gender equality and women's empowerment, inclusion of vulnerable groups as well as other cross-cutting issues and the SDGs, are included in the final evaluation report and the new project document.

The evaluation must provide factual information that is credible, reliable and useful. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators.

 As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. Travel to and in the country is constrained by  COVID-19. If it is not possible to travel to or within the country for the evaluation then the evaluator should develop a methodology that takes this into account the conduct of evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, survey and evaluation questionnaires. This should be detailed in the inception report and agreed with the Evaluation Manager. 

When the evaluation is to be carried out virtually, consideration should be taken for stakeholder availability, ability, or willingness to be interviewed remotely. In addition, their accessibility to the internet/computer may be an issue as many government and national counterparts may be working from home. These limitations and any others must be reflected in the evaluation report.

Evaluation products (deliverables)

An evaluation report and an associated power point presentation summarizing the findings of the evaluation and the proposed follow-up actions in a new UNDP Project Document format.

The consultant will be expected to deliver the following:

• Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
• Evaluation findings debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
• Draft evaluation report (60 pages including executive summary). The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide a set of comments to the evaluator within an agreed period of time, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in the evaluation guidelines.
•  
• Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how he/she has addressed comments.
•  
• Final evaluation report:
  • Executive summary
  • Introduction, including description of the work conducted
  • Findings and conclusions
  • Recommendations, including, as applicable, a revised work plan to address the pending tasks and eventual corrective action
  • Annexes providing a brief summary of the documents reviewed and persons interviewed with the description of the key content / conclusions drawn and any other relevant materials.

• Validation workshop for presentations to stakeholders and/or the evaluation reference group including GASTAT
• Evaluation brief and other knowledge products or participation in knowledge-sharing events, if relevant.

The consultant should present three hard copies of the report as well as an electronic copy. The draft final report should be submitted not later than three weeks after the end of the on-site mission and the final report within two weeks from receiving the comments of the project management and UNDP on the draft reports.

• Standard templates that need to be followed are provided in the Annexes section. It is expected that the evaluator will follow the UNDP evaluation guidelines and UNEG quality check list and ensure all the quality criteria are met in the evaluation report.

• In line with UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactory completed due to impact of COVID-19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her/their control.

Evaluation consultant required competencies

The consultancy will be conducted by one international evaluator.

The evaluator shall have prior experience in evaluating similar projects and have sufficient experience in statistics related to initiatives and main national and international development operations.

The evaluator selected should not have been involved in designing, executing, or advising any aspect of the project in order to avoid any conflict of interest with the project related activities.

The evaluator must present the following qualifications:

• Advanced university degree in Statistics
• Minimum 15 years of relevant professional experience in the area of statistics
• Familiar with UNDP approaches to development assistance
• Demonstrated understanding of issues related to human rights and gender; experience in gender sensitive evaluation and analysis in a development project; Previous experience with results‐based formulating, monitoring and evaluation methodologies;
• knowledge and/or experience of disability inclusion
• Strong communication and analytical skills
• Strong command of English language, both written and spoken
• Previous work experience in the region is an asset

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge