Background

The Kingdom of Saudi Arabia is the second largest state in the Arab world, with a reported population of 33,091,113 as of July 2018. A significant percentage of the nation’s inhabitants are immigrants seeking economic opportunity, making up 37% of the total Saudi population. Saudi Arabia has experienced a population explosion in the last 40 years and continues to grow at a rate of 1.63% per year. 

The economy of Saudi Arabia is one of the top twenty economies in the world (G20). It is dependent on oil as the country has the second-largest proven petroleum reserves, and it’s the largest exporter of petroleum in the world. It also has the fifth-largest proven natural gas reserves and is considered an "Energy Superpower". With a total worth of US$34.4 trillion, Saudi Arabia has the second most valuable natural resources in the world.

In 2016 the Saudi Government launched its Saudi Vision 2030 to reduce the country’s dependency on oil and diversify its economic resources and increase Foreign Direct Investment. Saudi Arabia has the largest economy in the Arab world. The petroleum sector accounts for roughly 87% of Saudi budget revenues, 90% of export earnings, and 42% of GDP Saudi Arabia's oil reserves and production are largely managed by the state-owned corporation Saudi Aramco. Another 40% of GDP comes from the private sector. The government has encouraged private sector growth for many years to lessen the kingdom's dependence on oil, and to increase employment opportunities for the swelling Saudi population. In recent decades the government has begun to permit private sector and foreign investor participation in sectors such as power generation and telecom and acceded to the WTO. During much of the 2000s, high oil prices enabled the government to post budget surpluses, boost spending on job training and education, infrastructure development, and government salaries.

As part of the public finance restructuring and in an effort to increase government revenues and consolidate fiscal sustainability through boosting non-oil revenues, VAT was imposed as of January 2018 at 5%. The “expat levy” is expected to generate by end 2018 SAR 28 billion (USD 7.46 billion). In addition to income from other taxes on imports and commerce and the Zakat. 

In attempt to diversify its economy and attract FDI, Saudi Arabia continued with plans to build mega projects on the Kingdom’s Red Sea coast line such as NEOM for innovation and technology and Amaala for wellness. The entertainment sector received a big boost in 2018. Movie theaters are established in the country and several musical and sports events were held throughout the year. This coincide with efforts to increase non-religious tourism to the country and ease of visa restrictions.

Efforts to nationalize “Saudization” of the job market are continuing to tackle the unemployment challenge, reaching 12.9% in 2018 with the aim to decrease it to 12.5% in 2019, especially among new university graduates.  Women were allowed to drive as of mid-2018. Thus, increasing women mobility and access to economic opportunities. Measures were put in motion to increase women participation in the labor market to 30% by 2030, such as easing up the male guardianship as a requirement, legislating an anti-harassment law and provision of cash subsidies for transportation.

The overall unemployment rate among Saudi women reached 31.1% (age 15 years old and above) in the 2nd quarter of 2018 compared to 7.6 among Saudi males. 

Based on the UN Common Country Strategic Framework (UNCCSF) developed by the UN Country Team in 2016, the UNDP CPD 2017-2021 was developed and approved in the same year. The Saudi Vision 2030 (SV 2030) was launched in June 2016 and the NTP 2020 subsequently developed with initiatives for each sector. The CPD is aligned with UNCCSF as per the following pillars:

Pillar 1. Sustainable economic and social development: the CPD outcome states:

“Pillar 1 will contribute to increasing the employment of qualified nationals (with a focus on youth and women) in productive sectors. UNDP will support the development of policies and strategies – as well as needed education reform – to meet the market demand, contributing towards a more diversified, knowledge-based economy. The achievement of UNDP efforts will be measured through the increase in numbers of youth accessing the labour market with appropriately adapted skills”.

UNDP projects and programme for the period are aimed at scaling-up its support for strategies and practices into development policies, plans and programmes. Interventions are focused on the promotion of a diversified and knowledge-based economy for the provision of jobs for the youth. At the institutional level, UNDP aims to strengthen the capacity of youth by ensuring they acquire the skills needed for the evolving markets.

Pillar 2. Public sector efficiency: the CPD outcome states:

“Local adaptation of the sustainable development goals should translate into the transparent and accountable provision of public goods and services. This is a crucial aspect of healthy socioeconomic development. Citizens’ access to high-quality basic services such as water and sanitation, health care, schooling, housing and transportation enhances their well-being. Access to sustainable transportation and telecommunications systems lowers transaction costs, leading to improvements in trade and economic activities”

UNDP plans to focus on capacity development for the efficiency, effectiveness, and accountability of public institutions and service delivery through outputs addressing various aspects of governance. It will provide assistance in improving public finance and public performance measurement systems and will help in the provision of capacity development to ensure innovative national research data-collection capacities and strengthen systems for evidence-based decision-making in the social sector. UNDP will promote the adherence of all partnerships to human rights principles, such as providing equal opportunities for men and women. UNDP also plans to continue to provide assistance in economic modelling as required, to improve the efficiency of the government subsidies system, and focus on the assistance provided to national partners to ensure sustainable, high-quality public services in an equitable and measurable manner, including key performance indicators and urban indicators.

Pillar 3. Sustainable natural resources management: the CPD outcome states:

“Sustainable development means that development is sensitive to the needs of people and the planet, in particular through environmentally sustainable extraction and use of natural resources. Sustainable economies must be supported by renewable resources and the sustainable management of resources through strong environmental policies, proper enforcement, and increased awareness, as well as the preservation of national heritage”.

UNDP will work with local and international partners to expand access to cleaner energy services and promote low-emissions technology. Achievements under this pillar will be measured by the degree to which proper energy-efficient and water conservation policies are being implemented and contribute to decreased energy and water consumption. Surveys used to gauge public and industrial awareness will be efficient tools to measure awareness level attained.

Duties and Responsibilities

Methodology

An overall guidance on evaluation methodology can be found in the UNDP Handbook on Monitoring and Evaluating for Results and the UNDP Guidelines for Outcome Evaluators.

The evaluation will be carried out by an independent evaluator and will engage a wide array of partners, stakeholders and beneficiaries. 

The evaluation is expected to take a “Theory of Change’’ (TOC) approach to determining causal links between the interventions that UNDP has supported, and observed progress in Sustainable Development, Public Sector Reform and Non-Oil Natural Resources Management initiatives at national in KSA. The evaluators will develop a logic model of how UNDP interventions in these areas are expected to lead to improved national transformation.

Evidence obtained and used to assess the results of UNDP support should be triangulated from a variety of sources, including verifiable data on indicator achievement, existing reports, evaluations and technical papers, stakeholder interviews, focus groups, surveys and site visits. 

The following steps in data collection are anticipated: 

Desk Review

A desk review should be carried out of the key strategies and documents underpinning the work of UNDP in KSA in support of Sustainable Development, Public Sector Reform and Non-Oil Natural Resources Management. This includes reviewing the UNCCSF, CPD and pertinent country programme documents AWPS, progress reports, monitoring and evaluation documents etc, to be provided by the UNDP Country Office.  

The evaluator is expected to review pertinent strategies, national plans and reports developed by the Kingdom of Saudi Arabia that are relevant to UNDPs support in Sustainable Development, Public Sector Reform and Non-Oil Natural Resources Management. 

Field Data Collection

Following the desk review, the evaluator will build on the documented evidence through an agreed set of field and interview methodologies, including:

•             Interviews with key partners and stakeholders

•             Field visits to project sites and partner institutions

•             Survey questionnaires where appropriate

•             Participatory observation, focus groups, and rapid appraisal techniques when needed

Methodological approaches may include some or all of the following:

Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments.

• Document review of all relevant documentation. This would include a review of inter alia
  • Project documents (contribution agreement).
  • Theory of change and results framework.
  • Programme and project quality assurance reports.
  • Annual workplans.
  • Results-oriented monitoring reports.
  • CO’s integrated work plan - IWP,
• Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organizations, UNCT members and implementing partners:
  • Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed. Based on the suggested questions mentioned above.
  • Key informant and focus group discussions with men and women, beneficiaries and stakeholders.
  • All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
• Surveys and questionnaires including participants in development programmes, UNCT members and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels.
• Field visits and on-site validation of key tangible outputs and interventions.
• The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries.
• Other methods such as outcome mapping, observational visits, group discussions, etc.
• Data review and analysis of monitoring and other data sources and methods.
  • Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources.

The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators.

Evaluation products (deliverables)

The terms of reference should clearly outline the outputs UNDP expects from the evaluation team as well as a detailed timeline and schedule for completion evaluation products. Where relevant, the TOR should also detail the length of specific products (pages).

• Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
• Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
• Draft evaluation report (within an agreed length). The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period of time, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines.
• Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
• Final evaluation report.

The suggested table of contents of the evaluation report is as follows:

I.             Title

II.           Table of Contents

III.          List of Acronyms and Abbreviations

IV.          Executive Summary

V.           Introduction

VI.          Description of the interventions

VII.         Evaluation Scope and Objectives

VIII.        Evaluation approach and methods

IX.          Data Analysis

X.            Findings and conclusions

XI.          Recommendations

XII.         Lessons Learned

XIII.        Annexes

Five working days following the contract signing, the evaluator will produce an inception report containing the proposed theory of change for UNDPs of the CDP outcomes.  The inception report should include an evaluation matrix presenting the evaluation questions, data sources, data collection, analysis tools and methods to be used.   The inception report should detail the specific timing for evaluation activities, deliverables, and propose specific projects visits and stakeholders to be interviewed.  The inception report will be discussed and agreed with the UNDP Country Office before the evaluator proceeds with meetings. The evaluator will also propose a rating scale in order that Performance rating will be carried out for the four evaluation criteria: relevance, effectiveness, efficiency, impact and sustainability

The draft evaluation report will be shared with stakeholders, and presented in a validation workshop, that the UNDP Country Office will organise. Feedback received from these sessions should be taken into account when preparing the final report.

 A length of 40 to 60 pages including executive summary is suggested.

Competencies

Evaluation required competencies 

The CPD evaluation will be undertaken by one external evaluator.

Required Qualifications of the International Evaluator:

• An advanced degree in economics, public administration, regional development/planning or any other social sciences related to economic management and pro-poor development;
• Vast experience in conducting Output/outcome/impact/CPD/UNDAF (UNCCSF) evaluations.
• Strong working knowledge of UNDP and its mandate, the civil society and working with government authorities;
• Extensive knowledge of results-based management evaluation, as well as participatory M&E methodologies and approaches;
• Experience in applying SMART (S Specific; M Measurable; A Achievable; R Relevant; T Time-bound) indicators and reconstructing or validating baseline scenarios;
• Extensive professional experience in the area of development, including gender equality and social policies;
• Strong reporting and communication skills; excellent communication skills with various partners including donors;
• Knowledge on mainstreaming Gender and Human rights in projects and programmes; and,
• Evidence of similar evaluations conducted. Previous experience on UNDP output/outcome/impact/CPD/UNDAF evaluations.
• Work experience in the region is an asset.
• Fluency in English. Knowledge of Arabic language is an advantage.

The Consultant will have overall responsibility for the quality and timely submission of the final evaluation report and will perform the following tasks:

• Manage the evaluation mission;
• Design the detailed evaluation scope and methodology and approach;
• Conduct the CPD evaluation in accordance with the proposed objective and scope of the evaluation and UNDP evaluation guidelines;
• Draft and present the Inception Report, the Draft and Final evaluation report;
• Finalize the evaluation report with recommendations and submit it to UNDP Country Office

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

In particular, the evaluator must be free and clear of perceived conflicts of interest. To this end, interested consultants will not be considered if they were directly and substantively involved, as an employee or consultant, in the formulation of UNDP strategies and programming relating to the CPD under evaluation.  The code of conduct and an agreement form to be signed by the consultant are included in Annex 1. 

Implementation arrangements

The UNDP KSA country office will select the evaluation consultant and will be responsible for the management of the evaluator. UNDP will designate a Focal Point for the evaluation and any additional staff to assist in facilitating the process (e.g., providing relevant documentation, arranging visits/interviews with key informants, etc.). The Country Office will take responsibility for the approval of the final evaluation report. The Assistant Resident Representative Programme will arrange introductory meetings within UNDP and Unit Heads to establish initial contacts with government partners and project staff. The consultant will take responsibility for setting up meetings and conducting the evaluation, subject to advanced approval of the methodology submitted in the inception report. The UNDP country office will develop a management response to the evaluation within two weeks of report finalization.

The Focal Point will collect feedback to enhance the quality of the evaluation. The Focal point will review the inception report and the draft evaluation report to provide detail comments related to the quality of methodology, evidence collected, analysis and reporting. The Focal Point will also advise on the conformity of evaluation processes to the UNEG standards. The evaluator is required to address all comments received completely and comprehensively. The Evaluator will provide a detail rationale to the Focal Point for any comment that remain unaddressed. 

While the Country Office will provide some logistical support during the evaluation, for instance assisting in setting interviews with senior government officials, it will be the responsibility of the evaluator to logistically and financially arrange their travel to and from relevant project sites. Planned travels and associated costs will be included in the Inception Report and agreed with the Country Office.

Required Skills and Experience

Education:

• Advanced degree in related fields.

Experience:

• At least 10 years of working experience, and considerable knowledge of material analyses and the ability to communicate results of research findings;
• Experienced in producing corporate publications;
• Previous projects or research.

Language:  

• Fluency in speaking and writing English and Arabic.