|Overview : |
Final Evaluation TOR
Advisory Services to the General Commission for Survey
General Authority for Survey & Geospatial Information
- Background and context
This project being evaluated represents the second phase in a strategic partnership focusing on promotion of the surveying works and the myriad products of surveying. The key objective of this intervention is to maintain provision of advisory services in developing the national capacities for effective geo-spatial surveys, generating multi-purpose knowledge from such surveys to efficiently boost national efforts in achieving the key directions of the Saudi Vision 2030 as well as promoting the national implementation of the Sustainable Development Goals (SDGs).
The completion of the previous phase has been instrumental in nurturing the idea of developing the foundation of financial sustainability whereby the General Commission of Survey (GCS) will achieve a high level of efficiency in all its hydrological and geospatial products. It is through this quality and efficiency that the GCS wishes to expand marketing of its products and to broaden the public awareness about its products and services. The core around which all aspects of this project revolve is the design and operationalizing of a National Centre for Geospatial Data.
On this basis, the project has four interconnected outputs:
1) National Centre for Geospatial Data established and operationalized
2) National capacities developed
3) Advisory services provided towards achievement of financial sustainability and contribution to the national economy
4) Advocacy promoted for a wider visibility of GCS
Capacity development will also focus on training in the core areas of land survey; geodetic survey; topographic survey’ and Cadastral survey.
In 2020, the mandate of GCS was revised, and the authority renamed the general Authority of Survey and Geospatial Information (GASGI) and the mandate under the old GCS changed to become more regulatory and less implementing. Article (4) of the Statute of the General Authority for Survey and Geospatial Information stipulates the following:
The Authority shall regulate, develop, supervise, and monitor the Sector in the Kingdom, except for activities relating to the Ministry of Defense, in a manner that achieves quality, enhances performance, and maintains security, in coordination with relevant agencies. The Authority may undertake any measure it deems necessary to achieve its objectives, including the following:
- Setting and updating rules and standards relating to the Sector to achieve integrated use of the geospatial information system among relevant agencies and to ensure compliance with said rules and standards.
- Proposing relevant laws upon obtaining the approval of the Board.
- Approving and developing the national geospatial infrastructure, national geodetic reference, national geodetic networks, and marine hydrographic survey; providing and marketing Sector-related data, products, services, electronic applications, topographic and aerial maps, and marine navigational charts; and maintaining their security and confidentiality.
- Developing and implementing strategic plans and conducting Sector-related studies and research in collaboration with relevant agencies.
- Licensing Sector activities, overseeing the training and classification of practitioners, and setting relevant rules.
- Protecting the interests of Sector stakeholders.
- Developing the Sector to attract investments and enhancing its growth.
- Building national capacities in the Sector, in cooperation with universities and specialized institutes within the Kingdom and abroad.
- Coordinating and cooperating with its counterparts in other countries as well as relevant international agencies and organizations, in accordance with statutory procedures.
- Representing the Kingdom locally and internationally, in accordance with statutory procedures.
- Supervising the publication and updating of the Kingdom’s Atlases and developing them in coordination with relevant agencies.
- Providing information, studies, and consultations on the Kingdom's land and maritime borders, and developing a database for such purpose in coordination with relevant agencies.
- Providing Sector-related consultations and services to government and private agencies, and other entities within the Kingdom and abroad
Advisory Services to the General Commission for Survey
Corporate outcome and output
Outcome: Improved knowledge-based equitable and sustainable development, underpinned by innovation and improved infrastructure
Output: National Policies developed to promote economic diversification with increased employment of Nationals
Date project document signed
14 May 2018
1 May 2018
31 December 2021
Project expenditure at the time of evaluation
The General Authority for Survey & Geospatial Information
- Evaluation purpose, scope and objectives
This evaluation is the final evaluation for the afore mentioned project and comes after a 9 months extension of the project aimed at re-positioning the project to help GASGI meet its new mandate. The evaluation and ensuing recommendations will help build a new project document serving GASCI better deliver its intended task and learn lessons from previous activities.
Scope and objectives of the evaluation include:
- Reviewing of the status of delivery of outputs as stated in the original project document:
- National Centre for Geospatial Data established and operationalized
- National capacities developed
- Advisory services provided towards achievement of financial sustainability and contribution to the national economy
- Advocacy promoted for a wider visibility of GCS
- Bottlenecks that may have impeded the delivery of the above
- Opportunities that may have been missed
- Opportunities that present themselves under the new mandate
Issues relate directly to the questions the evaluation must answer so that users will have the information they need for pending decisions or action. An issue may concern the relevance, coherence, efficiency, effectiveness or sustainability of the intervention. In addition, UNDP evaluations must address how the intervention sought to mainstream gender in development efforts, considered disability issues and applied the rights-based approach.
- Evaluation criteria and key guiding questions
Evaluation questions define the information that the evaluation will generate. This section proposes the questions that, when answered, will give intended users of the evaluation the information they seek in order to make decisions, take actions or increase knowledge. Questions should be grouped according to the four or five OECD-DAC evaluation criteria: (a) relevance; (b) coherence; (c) effectiveness; (d) efficiency; and (e) sustainability (and any other criteria used).
Project evaluation sample questions:
- To what extent was the project in line with national development priorities, country programme outputs and outcomes, the UNDP Strategic Plan, and the SDGs?
- To what extent does the project contribute to the theory of change for the relevant country programme outcome?
- To what extent were lessons learned from other relevant projects considered in the design?
- To what extent were perspectives of men and women who could affect the outcomes, and those who could contribute information or other resources to the attainment of stated results, taken into account during project design processes?
- To what extent does the project contribute to gender equality, the empowerment of women and the human rights-based approach?
- To what extent has the project been appropriately responsive to political, legal, economic, institutional, etc., changes in the country?
- To what extent did the project contribute to the country programme outcomes and outputs, the SDGs, the UNDP Strategic Plan, and national development priorities?
- To what extent were the project outputs achieved, considering men, women, and vulnerable groups?
- What factors have contributed to achieving, or not, intended country programme outputs and outcomes?
- To what extent has the UNDP partnership strategy been appropriate and effective?
- What factors contributed to effectiveness or ineffectiveness?
- In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
- In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?
- What, if any, alternative strategies would have been more effective in achieving the project objectives?
- Are the project objectives and outputs clear, practical and feasible within its frame? Do they clearly address women, men and vulnerable groups?
- To what extent have different stakeholders been involved in project implementation?
- To what extent are project management and implementation participatory, and is this participation of men, women and vulnerable groups contributing towards achievement of the project objectives?
- To what extent has the project been appropriately responsive to the needs of the national constituents (men, women, other groups) and changing partner priorities?
- To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights?
- To what extent was the project management structure as outlined in the project document efficient in generating the expected results?
- To what extent were resources used to address inequalities in general, and gender issues in particular?
- To what extent have the UNDP project implementation strategy and execution been efficient and cost-effective?
- To what extent has there been an economical use of financial and human resources? Have resources (funds, male and female staff, time, expertise, etc.) been allocated strategically to achieve outcomes?
- To what extent have resources been used efficiently? Have activities supporting the strategy been cost-effective?
- To what extent have project funds and activities been delivered in a timely manner?
- To what extent do the M&E systems utilized by UNDP ensure effective and efficient project management?
- Are there any financial risks that may jeopardize the sustainability of project outputs affecting women, men and vulnerable groups?
- To what extent will targeted men, women and vulnerable people benefit from the project interventions in the long-term?
- To what extent will financial and economic resources be available to sustain the benefits achieved by the project?
- Are there any social or political risks that may jeopardize sustainability of project outputs and the project contributions to country programme outputs and outcomes?
- Do the legal frameworks, policies and governance structures and processes within which the project operates pose risks that may jeopardize sustainability of project benefits?
- To what extent did UNDP actions pose an environmental threat to the sustainability of project outputs, possibly affecting project beneficiaries (men and women) in a negative way? What is the chance that the level of stakeholder ownership will be sufficient to allow for the project benefits to be sustained?
- To what extent do mechanisms, procedures and policies exist to allow primary stakeholders to carry forward the results attained on gender equality, empowerment of women, human rights and human development?
- To what extent do stakeholders (men, women, vulnerable groups) support the project’s long-term objectives?
- To what extent are lessons learned documented by the project team on a continual basis and shared with appropriate parties who could learn from the project?
- To what extent do UNDP interventions have well-designed and well-planned exit strategies which include a gender dimension?
- What could be done to strengthen exit strategies and sustainability in order to support female and male project beneficiaries as well as marginalized groups?
Evaluation questions on cross-cutting issues
- To what extent have poor, indigenous and physically challenged, women, men and other disadvantaged and marginalized groups benefited from the work of UNDP in the country?
All evaluation criteria and evaluation questions applied need to be checked to see if there are any further gender dimensions attached to them, in addition to the stated gender equality questions.
- To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?
- Is the gender marker assigned to this project representative of reality?
- To what extent has the project promoted positive changes in gender equality and the empowerment of women? Did any unintended effects emerge for women, men or vulnerable groups?
- Were persons with disabilities consulted and meaningfully involved in programme planning and implementation?
- What proportion of the beneficiaries of a programme were persons with disabilities?
- What barriers did persons with disabilities face?
- Was a twin-track approach adopted? 
Evaluation should employ a combination of qualitative and quantitative evaluation methods and instruments. The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and male and female direct beneficiaries. Methodological tools and approaches may include:
- Document review. This would include a review of all relevant documentation, inter alia
- Project document (contribution agreement).
- Theory of change and results framework.
- Programme and project quality assurance reports.
- Annual workplans.
- Activity designs.
- Consolidated quarterly and annual reports.
- Results-oriented monitoring report.
- Highlights of project board meetings.
- Technical/financial monitoring reports.
- Interviews and meetings with key stakeholders (men and women) such as key government counterparts, donor community members, representatives of key civil society organizations, United Nations country team (UNCT) members and implementing partners:
- Semi-structured interviews, based on questions designed for different stakeholders based on evaluation questions around relevance, coherence, effectiveness, efficiency, and sustainability.
- Key informant and focus group discussions with men and women, beneficiaries and stakeholders.
- All interviews with men and women should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
- Surveys and questionnaires including male and female participants in development programmes, UNCT members and/or surveys and questionnaires to other stakeholders at strategic and programmatic levels.
- Field visits and on-site validation of key tangible outputs and interventions.
- Other methods such as outcome mapping, observational visits, group discussions, etc.
- Data review and analysis of monitoring and other data sources and methods. To ensure maximum validity, reliability of data (quality) and promote use, the evaluation team will ensure triangulation of the various data sources.
- Gender and human rights lens. All evaluation products need to address gender, disability, and human right issues, wherever applicable.
The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between UNDP, key stakeholders and the evaluator.
- Evaluation products (deliverables)
These products include:
- Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
- Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
- Draft evaluation report (within an agreed length). A length of 40 to 60 pages including executive summary is suggested.
- Evaluation report audit trail. The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period of time, as outlined in these guidelines. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
- Final evaluation report.
- Presentations to stakeholders
- Required competencies
- Required qualifications: Advanced degree in a relevant field, a minimum of ten years’ experience conducting/ managing evaluations and relevant knowledge of the field of survey and geospatial information
- Technical competencies: team leadership skills and experience, technical knowledge in UNDP thematic areas, with specifics depending on the focus of the evaluation, data analysis and report writing etc.
- Technical knowledge and experience: Gender and disability inclusion competencies preferable. Technical knowledge and experience in other cross-cutting areas such equality, disability issues, rights-based approach, and capacity development.
- Language skills required: fluent English (reading, writing and spoken). Arabic a plus
Evidence to be presented:
- work samples
To support claims of knowledge, skills and experience.
The TOR should explicitly demand evaluators’ independence from any organizations that have been involved in designing, executing, or advising any aspect of the intervention that is the subject of the evaluation.
- Evaluation ethics
Statement that evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’.
“This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees, and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.”
- Implementation arrangements
The section describes the specific roles and responsibilities of all involved in this evaluation:
- Evaluation commissioner: The Resident Representative
- Evaluation manager: Lead the evaluation process and participate in all of its stages - evaluability assessment, preparation, implementation, management and use. Ensure quality assurance and manage the ERC portal
- Fulfil the contractual arrangements under the TOR
- Develop the evaluation inception report, including an evaluation matrix and a gender responsive methodology, in line with the TOR, UNEG norms and standards and ethical guidelines
- Conduct data collection and field visits according to the TOR and inception report
- Produce draft reports adhering to UNDP evaluation templates, and brief the evaluation manager, programme/ project managers and stakeholders on the progress and key findings and recommendations
- Consider gender equality and women’s empowerment and other cross-cutting issues, check if all and respective evaluation questions are answered, and relevant data, disaggregated by sex, is presented, analysed and interpreted
- Finalize the evaluation report, incorporating comments and questions from the feedback/ audit trail. Record own feedback in the audit trail including those of the members of the team, the evaluation manager, the commissioning programme unit and key stakeholders.
- Project manager:
- Provide inputs/ advice to the evaluation manager and evaluation reference group on the detail and scope of the TOR for the evaluation and how the findings will be used
- Ensure and safeguard the independence of evaluations
- Provide the evaluation manager with all required data (e.g. relevant monitoring data) and documentation (reports, minutes, reviews, studies, etc.), contacts/ stakeholder list etc.
- Ensure that data and documentation in general, but in particular related to gender equality and women’s empowerment and other cross-cutting issues, are made available to the evaluation manager
- Provide comments and clarification on the TOR, inception report and draft evaluation reports
- Respond to evaluation recommendations by providing management responses and key actions to all recommendations addressed to UNDP
- Ensure dissemination of the evaluation report to all the stakeholders including the project board
- Implement relevant key actions on evaluation recommendations
- Time frame for the evaluation process
This section lists and describes all tasks and deliverables for which the evaluator will be responsible and accountable, as well as those involving the commissioning office (e.g. workplan, agreements, briefings, draft report, final report).
- Desk review.
- Briefings of evaluator.
- Finalizing the evaluation design and methods and preparing the detailed inception report.
- In-country data collection and analysis (visits to the field, interviews, questionnaires).
- Preparing the draft report.
- Stakeholder meeting and review of the draft report (for quality assurance).
- Incorporating comments and finalizing the evaluation report.
In addition, the evaluator may be expected to support UNDP efforts in knowledge sharing and dissemination.
Example of working day allocation and schedule for an evaluation (outcome evaluation)
ESTIMATED # OF DAYS
DATE OF COMPLETION
Phase One: Desk review and inception report
Meeting briefing with UNDP (programme managers and project staff as needed)
At the time of contract signing
3 October 2021
UNDP or remote
Evaluation manager and commissioner
Sharing of the relevant documentation with the evaluator
At the time of contract signing
3 -10 October 2021
Evaluation manager and commissioner
Desk review, Evaluation design, methodology and updated workplan including the list of stakeholders to be interviewed
Within two weeks of contract signing
10 -15 October 2021
Submission of the inception report
(15 pages maximum)
Within two weeks of contract signing
3-18 October 2021
Comments and approval of inception report
Within one week of submission of the inception report
19 – 26 October 2021
Phase Two: Data-collection mission
Consultations and field visits, in-depth interviews, and focus groups
Within four weeks of contract signing
26 - 31 October 2021
With field visits
UNDP to organize with local project partners, project staff, local authorities, NGOs, etc.
Debriefing to UNDP and key stakeholders
1 November 2021
Phase Three: Evaluation report writing
Preparation of draft evaluation report (50 pages maximum excluding annexes), executive summary (4-5 pages)
Within three weeks of the completion of the field mission
2 -8 November
Draft report submission
^ Back to top