Background

São Tomé and Príncipe is a politically stable democracy and Small Island Developing State (SIDS), situated in the Gulf of Guinea, off the western equatorial coast of Central Africa. It comprises an archipelago of two main islands, São Tomé and Príncipe, situated about 140 km apart.

It’s population of 215,000, has grown, on average, by 2.17% per annum over the last decade, and is highly urbanised with 72.8% of the population living in towns and cities, and 40% living in the district of Água Grande in the urban sprawl of the capital city on the island of São Tomé. By contrast, the Autonomous Region of Príncipe hosts a population of just less than 9,000.

Just over half of STP’s population is female (50.5%) and more than one third of households are headed by women. Moreover, STP has a youthful population with 70% aged between 0 and 29 and 61% under the age of 24 (INE, 2012) which, if carefully managed, could create the potential for a demographic dividend.

Notable progress has been achieved in terms of human development in recent years, especially with regard to health and education indicators. STP’s score in UNDP’s Human Development Index (HDI) rose from 0.542 to 0.609 between 2010 and 2018 (UNDP, 2019), placing the country above average for Sub-Saharan Africa (0.537), but below the average for countries in the average human development group (0.645). These improvements are largely attributable to an increase in average life expectancy from 67.4 years in 2010 to 70.2 years in 2018, improvement in GNI per capita from $2,567 in 2010 to $3,024 in 2018, and an increase in the expected and average years of schooling from 10.6 to 12.7 and from 4.9 to 6.4 respectively over the period 2010 to 2018 (UNDP, 2019). These positive developments gains have led the country to be enlisted for LDC graduation status by 2024.

Yet STP still confronts a number of challenges to achieving the SDGs and an economic growth that has not been sufficiently inclusive. Lack of decent employment opportunities, particularly for women and young people, and rising inequality are two of the country’s greatest challenges. When adjusted for inequality, STP’s HDI drops by 16.7% (UNDP, 2019) and the country’s GINI coefficient has risen from 32.1 in 2000 to 56.3 in 2017, indicating an alarming widening in the inequality gap (World Development Indicators, 2020). Poverty rates have remained stubbornly high reducing marginally from 68.4% to 66.7% between 2010 and 2017. The 2017 Household Survey recorded the incidence of extreme poverty at 47% (INE, 2020). Some 46% of households comprising couples with children are poor, and 23% of households composed of extended families. Female-headed households are poorer than their male equivalents with a poverty rate of 61.6% compared to 55.8%. (INE, 2020). Urban areas and southern districts, such as Caué and Lembá, have higher levels of poverty incidence.

Severe food insecurity is a concern with around 10% of families reporting in 2017 that at least one family member had had to skip a full day of meals due to lack of money. And it appears this problem, due to seasonality, is not limited to the poor: 7.5% of non-poor families also reported a similar situation. Not having enough money for food seems to be a recurring problem with 42% of families reporting experiences of food shortages for a few months of the year, and 26% declaring that they are affected by this problem for almost the entire year.

Social protection programmes aimed at the poorest and most vulnerable groups are inadequately resourced and often unable to make timely and regular cash transfers to beneficiaries. In 2016, less than 0.65% of GDP was budgeted for social protection and social assistance programmes, significantly below the regional average of 1.2% (World Bank, 2018). Expanding these programmes to reach all poor households in STP would require expenditure of approximately US$7.2 million, or 2% of GDP. In addition to the lack of funding, sector policies are poorly coordinated and lack a common set of tools to serve those most at risk of being left behind.

STP’s economic challenges are typical of a SIDS and affect its ability to deal with shocks and achieve balanced budgets. The limited labour pool prevents the efficient production of goods and services at a scale needed to meet local and export market demand. Its insularity?and limited transport connectivity increase?imports and export costs, and the limited availability of land, and a small and largely unskilled workforce, prevent the country from diversifying its economy, making it more vulnerable to trade shocks. The economy is principally driven by agriculture, tourism, and foreign direct investment, and especially by government expenditures and investments. Socio-economic development is fragile and 97% of public investment budget is (on average) financed through debt and external aid. The economy is also overly dependent on trade and services (accounting for 70% of GDP), with tourism alone accounting for 65% of total exports. Paradoxically, and despite its potential, agriculture contributes barely 10% to GDP, principally through the production and export of cocoa which on average accounts for 90% of agricultural export earnings. However, although agriculture’s contribution to GDP is small, the sector is of strategic importance in socioeconomic terms given that it accounts for more than 70% of rural employment.

In order to control inflation, STP pegged its national currency (the Dobra) to the Euro in 2009 which has significantly contributed to price stability. Inflation declined to 3.96% in 2015 but has increased since spiking at 9% in 2018 due to a supply shock connected to locally produced food. In order to safeguard the exchange rate regime, the authorities have implemented prudent monetary and fiscal policies to keep international reserves at the necessary level.

Duties and Responsibilities

Evaluation scope and objectives

The CPD evaluation will focus on the formal UNDP country programme approved by the Executive Board (2017-2022. The scope of the CPD evaluation includes the entirety of UNDP’s activities at the outcome and output levels covering from 2017 to date. The evaluation covers interventions funded by all sources, including core UNDP resources, donor funds and government funds. Initiatives from regional and global programmes will be included in the CPD evaluation. The evaluation will also examine UNDP’s contribution toward cross-cutting issues, e.g. human rights, gender, leaving no one behind, and capacity development.  The evaluation should be forward-looking by drawing lessons from the current CPD and propose recommendations for the next CPD.

Evaluation criteria and key guiding questions

The evaluation will answer three broad questions as follows:

• What did the UNDP country programme intend to achieve during the period under review?
• To what extent has the programme achieved (or is likely to achieve) its intended objectives at the output level, and what contribution has it made at the outcome level and towards the UN Partnership Framework?
• What factors contributed to or hindered UNDP’s performance and eventually, the sustainability of results?

In addition to the above questions, the evaluation is expected to produce answers surrounding the of relevance, effectiveness, efficiency and sustainability of the country programme. Below are guiding questions. This evaluation will also include a special thematic evaluation of the Sustainable development and resilience to climate change theme & UNDP’s engagement in the same. Guiding questions for the thematic evaluation are listed in the Annex C.

Relevance

• To what extent has the current UNDP programme supported the government of STP in achieving the national development goals and implementing the 2030 Agenda for sustainable development? 
• To what extent has the UNDP programme responded to the priorities and the needs of target beneficiaries as defined in the programme document?
• Is UNDP perceived by stakeholders as a strong advocate for improving Health, Governance, and sustainable development and resilience to climate change in STP?
• Have the efforts made by UNDP and national partners to mobilize resources and knowledge been in line with the current development landscape?
• To what extent did the UNDP programme promote SSC/Triangular cooperation?
• Has UNDP been able to effectively adapt the programme to the effects of the COVID-19 pandemic in STP?

Effectiveness    

• By reviewing the programme results and resources framework, is the UNDP programme on track to achieve intended results at the outcome and output levels? What are the key achievements and what factors contributed to the achievements or non-achievement of those results?
• By examining the small-size initiatives funded by UNDP regular sources, how have these projects fulfilled their objectives? What are the factors (positive and negative) that contribute to their success or shortcomings? Are there recommendations or lessons that can be drawn from this approach?
• To what extent has UNDP programme contributed towards an improvement in national government capacity, including institutional strengthening? How could UNDP enhance this element in the next UNDP programme?
• Which programme areas are the most relevant and strategic for UNDP to scale up going forward?

Efficiency

• To what extent has there been an economical use of resources (funds, human resources, time, expertise, etc.)? What are the main administrative constraints/strengths?
• Is the results-based management system operating effectively and is monitoring data informing management decision making?
• To what extent has UNDP been efficient in building synergies and leveraging with other programmes and stakeholders in STP?
• How well does the workflow between UNDP and national implementing partners perform?
• To what extent have programme funds have been delivered in a timely manner?
• When UNDP provides implementation support services as per MOU with an implementing partner, how well has UNDP performed?

Sustainability

• What outcomes and outputs have the most likelihood of sustainability and being adopted by partners and why?
• To what extent do national partners have the institutional capacities, including sustainability strategies, in place to sustain the outcome-level results?
• To what extent are policy and regulatory frameworks in place that will support the continuation of benefits?
• To what extent have national partners committed to providing continuing support (financial, staff, aspirational, etc.)?
• To what extent do partnerships exist with other national institutions, NGOs, United Nations agencies, the private sector and development partners to sustain the attained results?

Human rights

• What barriers have been seen to the inclusion of vulnerable groups in UNDP’s work and what can be done to improve inclusion of these groups?

Gender Equality

• To what extent have gender equality and the empowerment of women been addressed in the programme strategic design, implementation and reporting? Are there key achievements?
• In what way could UNDP enhance gender equality in the next country programme?

An important note: Based on the above analysis, the evaluators are expected to provide overarching conclusions on achievement of the 2017-2022 CPD, as well as recommend key development priorities which shall inform the focus the new CPD. The evaluation is additionally expected to offer wider lessons for UNDP support in STP.

3.Methodology and approaches

The CPD evaluation methodology will adhere to the United Nations Evaluation Group (UNEG) Norms & Standards. The evaluation will be carried out by an independent evaluation team. The evaluation team should adopt an integrated approach involving a combination of data collection and analysis tools to generate concrete evidence to substantiate all findings. Evidence obtained and used to assess the results of UNDP support should be triangulated from a variety of sources, including verifiable data on indicator achievement, existing reports, evaluations and technical papers, stakeholder interviews, focus groups, surveys and site visits where/when possible.  It is expected that the evaluation methodology will comprise of the following elements:

• Review documents (Desk Review);
• Interviews with key stakeholders including government line ministries, development partners, civil society and other relevant partners through a participatory and transparent process;
• Consultations with beneficiaries through interviews and/ or focus group discussions;
• Survey and/ or questionnaires where appropriate;
• Triangulation of information collected from different sources/methods to enhance the validity of the findings.

The evaluation is expected to use a variety of data sources, primary, secondary, qualitative, quantitative, etc. to be extracted through surveys, storytelling, focus group discussions, face to face interviews, participatory methods, desk reviews, etc. conducted with a variety of partners. A transparent and participatory multi-stakeholder approach should be followed for data collection from government partners, community members, private sector, UN agencies, multilateral organizations, etc.

Evidence will be provided for every claim generated by the evaluation and data will be triangulated to ensure validity. An evaluation matrix or other methods can be used to map the data and triangulate the available evidence.

In line with the UNDP’s gender mainstreaming strategy, gender disaggregation of data is a key element of all UNDP’s interventions and data collected for the evaluation will be disaggregated by gender, to the extent possible, and assessed against the programme outputs/outcomes.

Special note:

Given the COVID 19 pandemic and the resultant restrictions may require many of the in-person missions / consultations and data gathering / activities to be carried out remotely using electronic conferencing means.  Alternatively, some or all in person interviews may be undertaken by the national consultant in consultation with the evaluation team leader.

4.Evaluation products (deliverables)

These products could include:

• Evaluation inception report (up to 10 pages). The inception report, containing the proposed the theory of change, and evaluation methodology should be carried out following and based on preliminary discussions with UNDP. The inception report should include an evaluation matrix presenting the evaluation questions, data sources, data collection, analysis tools and methods to be used. The inception report should detail the specific timing for evaluation activities and deliverables and propose specific site visits and stakeholders to be interviewed (this element can be shared with UNDP well in advance).  The inception report should be endorsed by UNDP in consultation with the relevant government partners before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluator. (see the inception report template in Annex H). 
• Kick-off meeting. Evaluators will give an overall presentation about the evaluation, including the evaluator team’s approach, work plans and other necessary elements during the kick-off meeting. Evaluators can seek further clarification and expectations of UNDP and the Government partner in the kick-off meeting.
• Evaluation debriefings. Immediately following the evaluation, the evaluation team is required to present a preliminary debriefing of findings to UNDP, key Government partners and other development partners.
• Draft evaluation report (max 60 pages including executive summary). UNDP and other designated government representative and key stakeholders in the evaluation, including the UNDP Bangkok Regional Hub, will review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period of time, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines.
• Evaluation report audit trail. Comments and changes by the evaluators in response to the draft report should be retained by the evaluators to show how they have addressed comments.
• Final evaluation report (see final evaluation template in the Annex I).
• A report on the sustainable development and resilience to climate change thematic evaluation (max 15 pages) by the assigned consultant; this paper will be presented as an appendix of the final report. The assigned consultant should integrate the important aspects of findings, recommendations and lessons learned into the final evaluation report.
• Evaluation brief (2 pages maximum) and other knowledge products or participation in knowledge-sharing events, if relevant.
• Evaluation Recommendations (see the management response in the Annex J)
• Presentations to stakeholders (this maybe done remotely)

5.Evaluation team composition and required competencies

The evaluation will be conducted by a team of three independent consultants comprising of:

• An Evaluation Team Leader (International);
• An Evaluation Member (international) focusing specifically on UNDP’s sustainable development and resilience to climate change portfolio; and
• A National Consultant who will provide knowledge of national context and support the full evaluation process as well as serve as an interpreter from Portuguese to English to and vice-versa when needed.

(a) Evaluation Team Leader (international), 39 working days

S/he has overall responsibility for conducting the CPD evaluation and providing guidance and leadership to the national consultant. In consultation with the team member, s/he will be responsible for developing a methodology for the assignment that reflects best practices and encourages the use of a participatory and consultative approach as well as delivering the required deliverables to meet the objective of the assignment. S/he will lead the preparation and revision of the draft and final reports, ensuring the assignments have been completed in the agreed timeframe.

S/he has responsibilities as follows:

• Leading the documentation review and framing of evaluation questions;
• Leading the design of monitoring and evaluation questions and field verification tools;
• Ensure efficient division of tasks between evaluation team members;
• Leading the evaluation team in planning, execution and reporting;
• Incorporating the use of best practice with respect to evaluation methodologies;
• Incorporating results from the governance thematic evaluation into the report;
• Responsible for and leading the drafting of inception report, finalization/quality control of the evaluation report including timely submission and adjustment;
• Leading the kick-off meeting and debriefing meeting on behalf of the evaluation team with UNDP and stakeholders;

The CO was granted a 1-year extension until December 2022 due COVID challenges. 

(b) International Evaluation Consultant, Sustainable Development and Resilience to Climate Change Area, 25 working days (Advertised and Recruited Separately)

S/he has overall responsibility for contributing to the CPD evaluation especially reviewing UNDP’s engagement in the Sustainable Development and Resilience to Climate Change outcome area. In consultation with the team leader, s/he will be responsible for developing a methodology for the assignment that reflects best practices and encourages the use of a participatory and consultative approach as well as delivering the required deliverables to meet the objective of the assignment. S/he will substantively contribute to the preparation and revision of the draft and final reports, ensuring the assignments have been completed in the agreed timeframe.   S/he will prepare a final report focusing on the findings, lessons learned and recommendations for UNDP’s future portfolio in this area. The key elements and highlights of Sustainable Development and Resilience to Climate Change will be integrated into the final country overall programme evaluation report.

S/he has responsibilities as follows:

• Contributing to the documentation review and framing of evaluation questions;
• Contributing to the design of monitoring and evaluation questions and field verification tools;
• Ensure efficient division of tasks between evaluation team members;
• Conducting the evaluation of the governance portfolio while contributing to the overall planning, execution and reporting;
• Incorporating the use of best practice with respect to evaluation methodologies;
• Contributing to the drafting of inception report, finalization/quality control of the evaluation report including timely submission and adjustment;
• Contributing to and participating in the kick-off meeting and debriefing meeting on behalf of the evaluation team with UNDP and stakeholders;

(c) National Evaluation Consultant, 39 working days) (Advertised and Recruited Separately)

S/he will support the Team Leader by providing knowledge of the development context in STP. S/he is well aware of STP cultural context and working with different government institutions; and when needed support as an interpreter between Portuguese and English. S/he collects all relevant documents and reports needed for the review.  S/he will support the team leader in coordinating with UNDP, government partners and other stakeholders.  S/he will play a crucial role in organizing meetings, workshops, interviews, consultations during the field missions. S/he will draft some parts of the report as assigned by the team leader. The consultant will advise the Team Leader on relevant aspects of the local context where the projects have operated.

Under the supervision of Evaluation Team Leader, s/he has responsibilities as follows:

• Support the documentation review and framing of evaluation questions;
• Support the coordination with UNDP, government partners, stakeholders and other parties; 
• Undertake field visits and collect feedback from beneficiaries, project stakeholders etc.;
• Support the Evaluation Team Leader and international consultant in planning, execution, analyzing and reporting;
• Incorporate the use of best practice with respect to evaluation methodologies;
• Support the drafting of inception report, finalization/quality control of the evaluation report;
• Participate and support the kick-off meeting and debriefing meeting with UNDP and stakeholders;
• Facilitate and support the field data collection in country;
• Translate the evaluation brief in STP language;
• Perform translation from English to STP and vice versa for the evaluation team when required.

Competencies

Core Competencies: 

• Promote the vision, mission and strategic objectives of UNDP.
• Demonstrate integrity towards models and ethical values of the UN.
• Have sensitivity and adaptability to the values and cultural differences, gender, religion, race, nationality and age.

Functional Competencies:

Technical Knowledge

• Demonstrate a good understanding in the International Cooperation field particularly on the mobilization and coordination of external aid;
• Demonstrate a good knowledge of issues relating to institutional development and the country's complex environment;
• Demonstrate ability to provide expert advice in the areas of strategic planning and policies;
• Demonstrate knowledge and experience in applying approaches based on human rights and promoting gender equality.

Development and Operational Effectiveness

• Ability to lead and build for domestic and international processes of strategic planning, processes of change, results-based management, work planning and reporting;
• Ability to understand, develop and manage budget, oversee the implementation monitoring and evaluation of development projects linked to the national system;
• Ability to apply development theory to specific context of the country to identify creative approaches and practices to overcome challenging situations;
• Ability to build and maintain effective partnerships with UN agencies and Mission and its main components, promote effectively, communicate sensitively with the different components;
• Ability to mobilize resources;
• Ability to implement new systems and behavioral and attitude changes.

Knowledge and learning management

• Promote knowledge sharing and learning culture;
• Have good knowledge of the UNDP program and operational issues and understanding of the UN integrated planning processes;
• Actively work for continued learning and personal development in one or more areas of their professional activity, acting on learning plans and apply newly acquired competences;
• Have knowledge of the program and policies of UNDP and UN in contexts of development during crises or weaknesses.

Management and leadership

• Build strong relationships with stakeholders and clients; focus on the impact and results for clients and respond positively to feedback and suggestions;
• Ability to establish effective working relations in a multi-cultural team environment;
• Excellent interpersonal skills;
• Attending to work consistently with energy and positive and constructive attitude;
• Demonstrate strong skills in oral and written communication;
• Demonstrate ability to communicate effectively, resource management; ability to plan and organize programs effectively;
• Demonstrate flexibility, initiative and insight;
• Demonstrate openness to change and ability to manage difficulties;
• Capacity to work under pressure and maintain a positive mood and outlook.
• Teambuilding skills and ability to work with diverse groups and facilitate dialogue.

Required Skills and Experience

Required Qualifications (Team Leader):

• Minimum Master’s degree in economics, public administration, regional development/planning or any other social sciences related to economic management and pro-poor development;
• 7 to 10 years relevant experience in undertaking evaluation in the development sector
• Strong knowledge of UNDP and its working approaches including partnership approaches with Government, civil society and community groups;
• Proven experience in conducting outputs/outcomes/impact/CPD/UNDAF evaluations;
• Experience in applying SMART (S Specific; M Measurable; A Achievable; R Relevant; T Time-bound) indicators and reconstructing or validating baseline scenarios;
• Demonstrated capacity in strategic thinking, problem solving and policy advice;
• Strong inter-personal skills, teamwork, analytical skills and organizational skills;
• Excellent presentation and drafting skills, and familiarity with information technology, including proficiency in word processing, spreadsheets, and presentation software;
• Fluency in Portuguese and English, both spoken and written;
• Previous experience working in STP or similar settings in the region is an advantage;
• Knowledge of the sensitivities of the context of STP is an asset.

Required Qualifications (International Consultant):

• Minimum Master’s degree in economics, public administration, regional development/planning or any other social sciences related to economic management and pro-poor development;
• 7 to 10 years relevant experience in undertaking evaluation in the development sector
• Extensive professional experience in the area of governance and sustainable development, including gender equality and social policies;
• Strong knowledge of UNDP and its working approaches including partnership approaches with Government, civil society and community groups;
• Proven experience in conducting outputs/outcomes/impact/CPD/UNDAF/thematic evaluations;
• Experience in applying SMART (S Specific; M Measurable; A Achievable; R Relevant; T Time-bound) indicators and reconstructing or validating baseline scenarios;
• Demonstrated capacity in strategic thinking, problem solving and policy advice;
• Strong inter-personal skills, teamwork, analytical skills and organizational skills;
• Excellent presentation and drafting skills, and familiarity with information technology, including proficiency in word processing, spreadsheets, and presentation software;
• Fluency in Portuguese and English, both spoken and written;
• Previous experience working in STP or similar settings in the region is an advantage;
• Knowledge of the sensitivities of the context of STP is an asset.

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ which are available here: http://www.unevaluation.org/document/detail/102. The consultants must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultants must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.

Please refer to ANNEX1 TOR for details.

Documents to be included when submitting the proposal:

• The following documents may be requested: Duly executed Letter of Confirmation of Interest and Availability using the template provided by UNDP;
• CV
• Technical proposal outlining the proposed methodology and approach for completing all the tasks outlined in the TOR.
• Financial Proposal that indicates the all-inclusive fixed total contract price, supported by a breakdown of costs, as per template provided. If an Offeror is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the Offeror must stipulate that arrangement at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP .

Financial Proposal (Please separate the financial proposal from the technical proposal and it should be incripted with a passport until when requested by UNDP). All submissions should be directed to a dedicated e-mail address: bidsSTP@undp.org

Lump-sum contracts

The financial proposal shall specify a total lump-sum amount, and payment terms around specific and measurable (qualitative and quantitative) deliverables (i.e. whether payments fall in instalments or upon completion of the entire contract). Payments are based upon output, i.e. upon delivery of the services specified in the TOR. In order to assist the requesting unit in the comparison of financial proposals, the financial proposal will include a breakdown of this lump-sum amount (including travel, living expenses, and number of anticipated working days).

Evaluation:

Cumulative analysis:

The proposals will be evaluated using the cumulative analysis method with a split 70% technical and 30% financial scoring. The proposal with the highest cumulative scoring will be awarded the contract. Applications will be evaluated technically, and points are attributed based on how well the proposal meets the requirements of the Terms of Reference using the guidelines detailed in the table below.

When using this weighted scoring method, the award of the contract may be made to the individual consultant whose offer has been evaluated and determined as:

a) Responsive/compliant/acceptable, and

b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

• Technical Criteria weighting; 70%
• Financial Criteria weighting; 30%

Only candidates obtaining a minimum of 49 points in the Technical Evaluation would be considered for the Financial Evaluation. Interviews may be conducted as part of technical assessment for shortlisted proposals.

Technical Evaluation criteria:

• Qualification and experience - 10 pts
• Technical approach as illustrated in the description of the proposed methodology - 15 pts
• Timeline reflecting proposed activities, which emphasis the ability to meet the proposed deadlines - 10 pts
• Evidence of experience of the consultant in conducting evaluations as detailed in the CV - 20 pts
• Reference from Past performance. To enable this reference check is carried out, applicants are required to provide a list of all related consultancies/ evaluations conducted during the past three years with associated contact details of references - 15 pts

For detailed ToR, letter of interest and confirmation of available, please kindly access the below link at the procument notice site.

UNDP | Procurement Notices - 87053 - CPD Evaluation Consultancy - two