Background

The Czech Republic and UNDP are long-standing partners with cooperation dating back to the late 1990s and taking a mature institutionalized form of the Czech UNDP Trust Fund in 2004.  

Since 2004, the Government of the Czech Republic and the UNDP in Europe and Central Asia have cooperated to bring the best practices and comparative knowledge to countries throughout the region. In 2018, the UNDP and the Czech Republic entered the next stage of collaboration – the Czech-UNDP Partnership for SDGs (CUP).  CUP is linked to UNDP 2018 – 21 Strategic Plan impact: To help countries eradicate poverty in all its forms and dimensions, accelerate structural transformation for sustainable development, and build resilience to crises and shocks. 

Project document defines the Project Output as follows: the Czech expertise and innovative technological solutions for SDGs in partner countries are available and applied in a sustainable manner. Contributing Outcome (RPD 2018–2021): RP OUTCOME 2: Addressing poverty and inequalities through more inclusive and sustainable development pathways Output 2.3. Enabling the environment strengthened through diverse partnerships to expand opportunities for the public and private sector, including alternative financing for the achievement of the SDGs Indicative Output(s) with gender marker: GEN 1.

The main goal of the Czech-UNDP Partnership for the SDGs (2018–2021) is to bring Czech expertise and innovative solutions for SDGs to partner countries. The?three components of the project include:??? 

• Expertise on Demand through which hands-on experience and trainings in the partner countries have been provided to facilitate the transfer of Czech expertise.? The Project Board decided to strengthen linkages of Expertise on Demand with the priority areas of Czech Development Cooperation.

• Challenge Fund: Czech solutions for SDGs have been providing scalable solutions for the identified development problems applicable at the country and sub-regional level. Since the?beginning of the?CUP there have been 5 rounds of Call for Applications and?50 innovative solutions have been applied since 2018. According to the 2020 Project Board decision, Challenge Fund thematic priorities were aligned with UNDP COs priorities to enable further synergies between project initiatives and COs ongoing or planned projects.?

• Knowledge management: to mobilize knowledge and know-how and feed it into resolving specific development objective. The project applies KM strategy to ensure that its activities contribute to broader outcomes and make an impact.?? 

Three priority countries (Bosnia and Herzegovina, Georgia and Moldova) were originally defined as partners for the CUP. Those countries are in need of attracting socially and environmentally sound investments that could solve concrete development problems and assure sustainability. The UNDP Regional Programme stresses the need to engage more strategically with Private sector within the Addis Ababa Action Agenda that calls for investments with positive development impact. The Development Cooperation Strategy of the Czech Republic 2018 – 2030 also emphasizes support for development-oriented investments in riskier markets. According to UNDP Strategic Plan, 2018 – 2021, “the private sector is a critical development partner.”  As such, private sector can become also a strategic partner and a source of technological solutions and investment for SDGs.

Since 2020, another 3 partner countries, Zambia, Ethiopia and Cambodia, have been added alongside with Moldova, Georgia and Bosnia and Herzegovina. The geographical extension has aligned CUP partner countries with the Czech ODA priority countries.

The CUP addresses development challenges identified as key by the UNDP COs in its three partner countries. At the same time, due to the wide range of development challenges being addressed by UNDP COs, CUP focuses on areas where the Czech Republic has substantive expertise and can offer added value and innovative solutions. As such, the CUP priorities are aligned as much as possible with the priorities of the Czech ODA in partner countries and its Programme documents. It will focus on the following four thematic areas:

•          Good governance and rule of law (SDG16)

•          Sustainable management of natural resources (SDG6, SDG13)

•          Economic transformation and development (SDG7, SDG8)

•          Agriculture and rural development (SDG2, SDG15)

The CUP is directly implemented by the UNDP’s Regional Hub for Europe and the CIS within the delegated direct Implementation authority for the Regional Programme implementation, in line with UNDP’s Programme and Operations Policies and Procedures. The UNDP Regional Hub acts as the Executive responsible for the overall management, backstopping and monitoring of the project. The project is implemented in close coordination with and involvement of the participating UNDP Country Offices. The project is managed by two project Specialists, the Senior project Specialist seconded by the Ministry of Foreign Affairs of the Czech Republic  and the Junior Project Specialist, UNV funded by the donor´s UNV programme. The annual budget of the project is 17 million CZK (750, 000 USD). The project donor is the Ministry of Foreign Affairs of the Czech Republic.

The CUP positions itself as a service line for UNDP COs, contributing to one of the key objectives of the UNDP which is to “help countries to build capacities to address complex and interconnected development challenges …”  It supports them in solving key development challenges by facilitating transfer of top-notch Czech expertise, know-how and innovative solutions. In doing so, it builds on previous success of the Czech – UNDP Trust Fund 2014 – 2017.

An independent 2014 - 2016 evaluation of the Czech – UNDP Trust Fund concluded, that the Fund managed to address the most important development needs identified by the national partners in partnership with UNDP COs, was successful in promoting Czech “know-how”. Its activities were aligned with geographic priorities of the Czech ODA and supportive of the overall programmatic framework and planned results of UNDP’s 2014-2017 Strategic Plan and Regional Programme document for Europe and CIS for 2014-2017. 

The current phase of the Czech-UNDP Partnership has been set for 2018-2021, however, the Czech MFA has already declared its interest to extend the Project document for 2022-4.

Duties and Responsibilities

Evaluation purpose, objectives and scope

Purpose

The purpose of the Mid-Term Review is to provide an impartial review of the Project in terms of its relevance, effectiveness, efficiency, impact, sustainability, management and achievements including impact of COVID-19 pandemic on the Project implementation of the Project activities. The information, findings, lessons learned and recommendations generated by the evaluation will be used by the Project Board, UNDP, and by the implementing partners to strengthen the project implementation for the phase of project extension for 2022-4.

Objective

The evaluation objective is to examine the overall performance of the Project, its results, inputs and activities; and how the outputs delivered positive changes in partner countries to strengthen cooperation in reducing poverty and achieving sustainable development goals and to what extend the Project succeeded to involve private sector as a source of technological solutions and investments for SDGs. In a substantive analysis of the effectiveness of the project approach and feedback from UNDP COs, innovators, targeted groups and beneficiaries, the evaluation should highlight strengths, weaknesses/gaps, good practices and impact of COVID-19 on the project and provide forward-looking actionable recommendations to the Project to enhance effectiveness of transfer of the Czech expertise and innovative solutions in partner countries.

Scope

The evaluation will assess the extent to which the specific project objective/outcome and results/outputs have been achieved since mid-2018 (based on the Project Document and results framework). The evaluation will look into all project activities and processes implemented in the partner countries.

Specifically, the evaluation will review and make recommendation regarding the implementation of the critical project’s aspects, such as strategies, implementation mechanisms and partnerships with the UNDP COs in transfer of the Czech expertise and innovative solutions.

Evaluation criteria and key questions (based on OECD DAC criteria)

The Project evaluation is to answer the following questions to determine the Project’s relevance, performance, results, effectiveness, efficiency, impact and sustainability, including lessons learned and forward-looking recommendations. The evaluation questions are summarized below.

Relevance

• Are the Project’s objectives relevant to the needs of the UNDP partner countries in terms of their social and economic development?
• To what extent is the Project aligned with the relevant national development priorities in the partner countries as the beneficiaries, UNDP strategic objectives and SDG 17 - partnership?
• To what extent is the Project responsive to the changing environment in country at national and subnational levels and how should it adapt to these changes?
• To what extent was the method of delivery selected by the Project appropriate to the development context?
• To what extent does the Project address need of marginalized groups and contribute to gender equality?

Effectiveness

• To what extent are the Project activities implemented and intended results and the specific objective/outcome achieved and reported? What are the main project achievements? Please provide outline of a measurable overview of the Project results against the indicators and their target values/statements as defined in the Project results framework.
• What are the positive or negative, intended or unintended, changes brought about by the Project’s interventions? This may, inter alia, include an overview of benefits the Project brought to beneficiary institutions and citizens in partner countries.
• What factors have contributed to achieving or not achieving the intended specific objective/outcome and outputs/results?
• To what extent has the Project managed to perform sharing of Czech knowledge, expertise and innovative solutions to the UNDP COs and their national partners?
• How COVID-19 affected or limited the Project activities and what actions were undertaken to offset the negative impact?
• Assess the degree to which project implementation was flexible and adaptive to the context.

Efficiency

• Have resources (financial, human, technical) been allocated strategically to achieve the Project results?
• Are there any weaknesses in the Project design, management, human resource skills, and resources?
• Analyse the role of the Project Board and whether it is optimally being used for decision making.
• Assess the timeline and quality of the reporting followed by the Project.
• Identify factors and constraints, which have affected the Project implementation including technical, managerial, organizational, institutional and socio-economic policy issues in addition to COVID-19 related challenges and other external factors unforeseen during the Project design.
• To what extent did Project engage or coordinate with beneficiaries, implementing partners, other United Nations agencies and national counterparts to achieve outcome-level results?
• Are the outcomes of Expertise on Demand and Innovation solution initiatives effectively communicated in the partner countries, in donor´s country and within the UNDP? Is there a communication strategy in place?

Impact

• What are the Project effects and impact in terms of implemented Project activities, both in qualitative, and quantitative terms, on achievement of specific development results by partner countries via transfer of the Czech expertise and innovative solutions.
• To what extent are key stakeholders/final beneficiaries satisfied with the Project implementation, specifically in terms of the partnership support and what are specific expectations for the potential follow-up assistance?

Sustainability

• To what extent the Project outputs/results are likely to be sustainable beyond the Project’s lifetime? How could the Project results be further sustainably projected and expanded in the countries?
• What would be future priority interventions to ensure long-term sustainability of the project’s achievements and contribute to further sharing knowledge and expertise to partner countries?
• How has the Project enhanced and contributed to the development of national capacity in partner countries and of the Czech experts and innovators?

Lessons learned

• Analyse the main lessons learned in relation to the effectiveness of implementation modalities.

Methodology

Based on the UNDP Evaluation Guidelines, UNEG Norms and Stand for Evaluations and in consultations with UNDP IRH the evaluation will be participatory, involving relevant stakeholders.

The International Evaluation Consultant (the Consultant) will propose an evaluation methodology and agree on a detailed plan for the assignment as a part of the evaluation Inception Report. The proposed methodology may employ any relevant and appropriate quantitative, qualitative or combined methods to conduct the Final Project Evaluation, exploring specific gender sensitive data collecting and analytical methods and tools applicable in the concrete case. The Consultant is expected to creatively combine the standard and other evaluation tools and technics to ensure maximum reliability of data and validity of the evaluation findings.

Standard UNDP evaluation methodology would suggest the following data collecting methods:   

• Desk review: The Consultant will conduct a detailed review of the programmatic materials and deliverables including the Project Document, Challenge Fund Guidelines, monitoring and project quality assurance reports, annual workplans, progress and annual reports etc. 
• Key informant interviews: The Consultant will interview representatives of main institutional partners, UNDP, innovators and their local partners, experts, representatives of targeted groups and final beneficiaries. For the interviews, the Consultant is expected to design evaluation questions around relevance, effectiveness, efficiency and sustainability criteria, according to different stakeholders to be interviewed. 
• Meetings via Zoom, Teams or WhatsApp with will be arranged to meet with beneficiaries and stakeholders and review the results of the Project;
• Other methodologies, as appropriate, such as surveys, case studies, statistical analysis, social network analysis, etc.

The COVID-19 global pandemic made travel to the countries impossible. In this case the evaluation team should develop a methodology that takes this into account the conduct of the evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. This should be detailed in the Inception report and agreed with the Evaluation Manager.

The expected duration of the assignment is up to 20 workdays in the period September –   October 2021.

The tentative Schedule of Activities and Deliverables

Deliverables

Competencies

Core values

• Demonstrates integrity and fairness by modelling UN values and ethical standards;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Core competencies

• Demonstrates professional competence to meet responsibilities and post requirements and is conscientious and efficient in meeting commitments, observing deadlines and achieving results;
• Results-Orientation: Plans and produces quality results to meet established goals, generates innovative, practical solutions to challenging situations;
• Communication: Excellent communication skills, including the ability to convey complex concepts and recommendations, both orally and in writing, in a clear and persuasive style tailored to match different audiences;
• Teamwork: Ability to interact, establish and maintain effective working relations with a culturally diverse team;
• Client orientation: Ability to establish and maintain productive partnerships with national partners and stakeholders and pro-activeness in identifying of beneficiaries and partners’ needs and matching them to appropriate solutions.

Required Skills and Experience

Academic Qualifications/Education

• Advanced university degree in international relations, social sciences, economics, public administration, law or other related sciences; MA in any of indicated fields is considered an advantage.

Experience

• At least 3 years of extensive expertise and experience in evaluations of the projects/programmes; 
• Sound knowledge of UNDP results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
• General understanding and knowledge of the political/administrative and development context in partner countries, with specific focus on project targeted beneficiaries and stakeholders;
• Proven analytical skills and ability to conceptualize and write concisely and clearly.

Languages Requirements

• Fluency in English
• Knowledge of Czech language is an asset

Other

• Excellent computer skills (MS Office applications) and ability to use information technologies as a tool and resource.

Evaluation of Applicants

Individual consultants will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal.

The award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:

a) responsive/compliant/acceptable, and

b) Having received the highest score out of a pre-determined set of weighted technical (P11 desk reviews and interviews) and financial criteria specific to the solicitation.

Only highest ranked candidates who would be found qualified for the job based on the P11 desk review will be invited for an interview”.

Financial

Technical Criteria - 70% of total evaluation – max. 70 points:

Criteria A (Relevant education) – max points: 5

Criteria B (Experience) – max points: 25

Criteria C (knowledge of English and Russian) – max points: 10

Criteria D – interviews (expertise in evaluation of projects/programmes will be scored and evaluated) – max 10

Criteria E - interview (knowledge of results-based management systems will be scored and evaluated) – max 10

Criteria F – interview (knowledge of development context of the CIS region will be scored and evaluated) – max 10

Financial Criteria - 30% of total evaluation – max. 30 points

The financial proposal will specify a total lump sum amount and payment terms around specific and measurable (qualitative and quantitative) deliverables. Payments are based upon delivery of the services specified in the TOR and according to the lump sum breakdown as presented below.

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG 'Ethical Guidelines for Evaluations'. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

TOR annexes (Application annexes can be found under the link: http://procurement-notices.undp.org/view_notice.cfm?notice_id=82175)

Annex 1. Project results framework

Annex 2. Indicative list of main stakeholders to be interviewed in evaluation

Annex 3. List of documents to be considered for the evaluation desk review

Annex 4. Evaluation matrix template

Annex 5. Standard outline for the UNDP evaluation report

Annex 1. Project Result Framework

(attached separately)

Annex 2. Indicative list of main stakeholders to be interviewed in evaluation

UNDP Country Offices and the national partners in Bosnia and Herzegovina, Georgia, Moldova, Ethiopia, Cambodia and Zambia

Accelerator Labs in Bosnia and Harzegovina and Ethiopia

Green City Lab in City of Chisinau

Selected experts that were engaged for the assignments under the Experts on Demand modality;

Selected innovators awarded within the Challenge Fund modality;

Local Partners of the Czech innovators;

Local institutions and other stakeholders as beneficiaries of the initiatives;

Czech Embassies in partners countries;

Ministry of Foreign Affairs of the Czech Republic (donor);

Czech Development Agency

Annex 3. List of documents to be considered for the evaluation desk review

1. ProDoc;
2. Challenge Fund Guidelines;
3. Expertise on Demand Guidelines;
4. Project Annual Progress Reports;
5. Project Quarterly Reports
6. Minutes of the Project Board meetings;
7. Evaluation reports;
8. Mission Reports BTRs);
9. Communications and Visibility Strategy
10. CUP website

Annex 4. Evaluation matrix template

Annex 5. Standard outline for the UNDP evaluation report

1. Title and opening pages with details of the project/project/outcome and of the evaluation team.

2. Project and evaluation information details: title, Atlas number, budgets and project dates and other key information.

3. Table of contents.

4. List of acronyms and abbreviations.

5. Executive summary: a stand-alone section of maximum four pages including the quality standards and assurance ratings.

6. Introduction and overview. What is being evaluated and why?

7. Description of the intervention being evaluated. Provides the basis for report users to understand the logic and evaluability analysis result, assess the merits of the evaluation methodology and understand the applicability of the evaluation results.

8. Evaluation scope and objectives. The report should provide a clear explanation of the evaluation’s scope, primary objectives and main questions.

9. Evaluation approach and methods. The evaluation report should describe in detail the selected methodological approaches, methods and analysis.

10. Data analysis. The report should describe the procedures used to analyse the data collected to answer the evaluation questions.

11. Findings and conclusions. Evaluation findings should be based on an analysis of the data collected and conclusions should be drawn from these findings. Specifically, the Project Evaluation Report will include a review of impact and effects of the Project on its beneficiary institutions.

12. Recommendations. The report should provide a reasonable number of practical, feasible recommendations directed to the intended users of the report about what actions to take or decisions to make prior and during the Project Phase III to be elaborated.

13. Forward-looking actionable recommendation for the Project, outlining key strategic priorities to be addressed in the potential next phase of the project.

14. Lessons learned. As appropriate and as requested in the TOR, the report should include discussion of lessons learned from the evaluation of the intervention.

15. Annexes.