Background

UNDP Istanbul Regional Hub (IRH) is UNDP’s main knowledge and advisory hub for the countries serviced by UNDP’s Regional Bureau for Europe and CIS (RBEC). The Hub implements the Regional Programme (2018-2021) that has been formulated based on the most crucial regional challenges in the region with a focus on a strengthened approach to specific sub-regional challenges. Transformative Governance and Finance Facility-II builds on the successes of and lessons learned from the first phase of the Transformative Governance and Finance Facility (2015-2017), and seeks to utilize new technologies, data and alternative finance mechanisms in support of achieving the Sustainable Development Goals (SDGs). The project is a joint-partnership with UNDP Istanbul Regional Hub and the Ministry of Finance of the Slovak Republic.   The second phase of TGFF started on 1 January 2018 and is expected to end as 31 December 2020. TGFF-II is implemented in the Europe and Central Asia (ECA) region mainly focusing on the following countries: Albania, North Macedonia, Serbia, Moldova, Armenia, Tajikistan, Montenegro, Kosovo , Georgia, Ukraine and Belarus. 

The overall objective of the TGFF-II is: 

Development, and in particular governance, outcomes in the region are improved, in furtherance of achieving the SDGs, by empowering various groups within and outside the public sector to leverage new technology, data and alternative finance mechanisms. 

To help achieve this objective aims to achieve three key outputs: 

• New technologies, data and engagement of citizens are used to improve governance and development outcomes. 
• Collaborative platforms are utilized at the national and regional level to drive innovative solutions to governance and development challenges.
• Effectiveness of development interventions is strengthened and enhanced through alternative finance mechanisms. 

Duties and Responsibilities

Evaluation Purpose, Scope and Objectives

As stated previously TGFF-II Project has started on 1st January 2018 and planned to be completed on 31 December 2020. As the project has reached the first half of its duration, mid-term evaluation is to assess if TGFF is functioning as planned in terms of efficiency, relevance, effectiveness and sustainability. The consultant will assess the performance of the Project in achieving the results and constraints. All three outputs of TGFF will be evaluated and countries took part in the implementation will be included in the evaluation.

The consultant is expected to identify weaknesses and strengths of the project design and implementation, and to come up with recommendations regarding the overall design and orientation of the project and on the work plan for the remaining period of the Project, after evaluating the adequacy, efficiency, and effectiveness of implementation, as well as assessing the achievements the project outputs and outcomes. The evaluation will also assess early signs of project success or failure and prompts adjustments. The results and recommendations of the evaluation would therefore help UNDP Project team to document lessons learnt and best practices for the next project cycle.

The scope of the mid-tern evaluation covers all activities undertaken in the indicated duration of the project. This refers to: 

• Planned outputs of the project compared to actual outputs and the actual results as a contribution to attaining the project objectives. 
• Problems and necessary corrections and adjustments to document lessons learnt. 
• Efficiency of project management, including the delivery of outputs and activities in terms of quality, quantity, timeliness and cost efficiency. 
• Likely outcomes and impact of the project in relation to the specified goals and objectives of the programme. 

The evaluation comprises the following elements: 

• Assess whether the project design was clear, logical and commensurate with the time and resources available; 
• An evaluation of the project’s delivery of achievement of its overall objectives; 
• An evaluation of project’s performance in relation to the indicators, assumptions and risks specified in the logical framework matrix and the project document; 
• Identification and, to the extent possible, quantification of any additional outputs and outcomes beyond those specified in the Project Document; 
• An evaluation of project coordination, management and administration. This includes specific reference to: a)  Organizational/institutional arrangements for collaboration among the different stakeholders involved in project arrangements and execution; b) The effectiveness of the monitoring and evaluation framework/mechanisms used in monitoring on a day to day basis, progress in project implementation; c) Administrative, operational and/or technical challenges and constraints that influenced the effective implementation of the project; d) An assessment of the functionality of the institutional structure established and the role of the Project Board; e) Financial management of the project, including the balance between expenditures on administrative and overhead charges in relation to those on the achievement of substantive outputs. 
• Progress towards sustainability and replication of project activities; 
• Assess the extent to which the design, implementation and results of the project have incorporated a gender equality perspective and human rights-based approach ;
• Assess of the extent to which the design, implementation and results of the project have incorporated the environmental sustainability concerns and make recommendation accordingly;
• Lessons learned during programme implementation; 
• Recommendations for improvement

Evaluation Criteria and Key Guiding Questions

The programme will be evaluated based on the DAC evaluation criteria: 

• Relevance: measures whether the programme addresses an important development goal and whether its objectives are still valid.
• Effectiveness: measures whether the programme activities achieve its goal.
• Efficiency:  measures the cost effectiveness, i.e. the economic use of resources to achieve desired results. 
• Sustainability: measures whether the benefits of the programme are likely to continue after donor funding has been withdrawn. The programme needs to be environmentally as well as financially sustainable.
• Impacts of intervention:  measure the positive and negative changes produced by the programme, directly or indirectly, intended or unintended.

More specifically, the evaluation aims to address the following questions for each evaluation criteria:

Relevance 

• What has been the main focus of the project implementation so far? Who are the main beneficiaries? How were they selected? 
• The extent to which the programme activities are suited to the priorities and policies of the target group, recipient and donor.
• To what extent did the objectives remain valid throughout the programme duration?
• Were the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
• Were the activities and outputs of the programme consistent with the intended impacts and Effects?
• Is the gender marker data assigned to this project representative of reality?

Effectiveness

• To what extent were the objectives achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?
• Did the activities contribute to the achievement of the planned outputs?
• In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
• In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?

Efficiency

• To what extent have resources been used efficiently? Have activities supporting the strategy been cost-effective?
• Were objectives achieved on time?
• To what extent was the project management structure as outlined in the project document efficient in generating the expected results?
• To what extent have the UNDP project implementation strategy and execution been efficient and cost-effective?

Sustainability

• Are there any financial risks that may jeopardize the sustainability of project outputs?
• To what extent will the benefits of the programme or programme continue after donor funding stops?
• What are the major factors which can influence the achievement or non-achievement of sustainability of the project?  
• What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or programme?  
• What is the level of stakeholders’ ownership for the project benefits to be sustained?
• Does the project have a clear exit strategy?

Methodology

The final decision on the specific design and methods for the evaluation will emerge from consultation among programme staff, the evaluators and key stakeholders, based on the inception report prepared by the evaluator, about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, given limitations of budget, time and data.

The evaluation should use a combined methods approach, drawing on both primary and secondary, quantitative and qualitative data to come up with an overall assessment backed by clear evidence. Data will be collected through surveys of all relevant stakeholders (national and local Government institutions, UNDP COs, development partners, beneficiaries, etc.) and other selected mechanisms (e.g, key informant interviews, focus group discussions and et cetera). Further data on the project indicators will be used by the evaluation to assess the project progress and achievements.  

The final methodological approach including interview schedule, possible field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators.

The evaluation methodology will include the following:

• Desk review of programme document, monitoring reports (such as minutes of the Board meeting, mission reports, and other internal documents including consultant and financial reports); 
• Review of specific products produced so far, including datasets, management and action plans, publications (e.g, blogs, media et cetera) and other material and reports;
• Interviews with the Project Manager, donor, consultants, relevant CO DRRs
• Interviews with other relevant stakeholders involved, namely the EBRD, etc. 
• Focus group discussions with all stakeholders

The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the project manager, implementing partners and direct beneficiaries.

Deliverables

This section presents the key evaluation products the evaluator will be accountable for producing. The deliverables are the following: 

• Evaluation inception report— (mid-March) An inception report should be prepared by the evaluator before going into the full-fledged data collection exercise. It should detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods, proposed sources of data and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product. The inception report provides the programme unit and the evaluator with an opportunity to verify that they share the same understanding about the evaluation and clarify any misunderstanding at the outset.  The inception report will be discussed and approved by the Regional Innovation Specialist 1 week after signing the contract.
• Preliminary Findings  - (beginning of April) Following field missions and prior to the drafting of the evaluation report, the evaluation team should debrief the UNDP project/programme and management teams with preliminary findings. This gives an opportunity to discuss preliminary findings and address any factual errors or misunderstandings, prior to writing the evaluation report.
• Draft evaluation report— (mid-April) Submission of draft evaluation report to UNDP for comments and inputs. The programme unit and key stakeholders in the evaluation will then review the draft evaluation report to ensure that the evaluation covers the scope and meets the required quality criteria.
• Presentation of Draft evaluation report (in April) (PPT presentation) to the Project Steering Committee for inputs and comments.
• Final evaluation report – (end of April) The final report should be completed 1 week after receipt of consolidated comments from stakeholders.

Payment (100%) will be done as a lump-sum upon confirmation of UNDP on delivering on the above-mentioned deliverables in a satisfactory manner.

Evaluation Ethics 

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation . The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

Evaluator should be independent from any organizations that have been involved in designing, executing or advising any aspect of the TGFF-II Project that is the subject of the evaluation.

Implementation Arrangements

This section describes the organization and management structure for the evaluation and defines the roles, key responsibilities and lines of authority of all parties involved in the evaluation process. Implementation arrangements are intended to clarify expectations, eliminate ambiguities, and facilitate an efficient and effective evaluation process. Evaluator will report to Regional Innovation Specialist who will be the evaluation manager as well for this evaluation.

UNDP Project Team

UNDP is responsible for the management of this final evaluation and will contract independent consultant to conduct the evaluation on behalf of the TGFF-II Project. Project team will be the focal point for the evaluation and will provide technical assistance during the evaluation process such as setting up interviews and payments for the consultant. Day-to-day management of the Evaluation Team will be provided by Project team. Project team will ensure that all issues pertaining to the contract with the Evaluation Team, including payments are completed on schedule and will be responsible for facilitating the work of the Evaluation Team. Project team will provide all documentation to the team for the desk review, facilitate interview appointments and field visits and convene focus group meetings.

Time Frame for the Evaluation Process

The evaluation will be conducted in 1 March – 30 April 2020 for an estimated 20 working days. The evaluation will include the following phases with their respective time frame.

Desk Review and Inception report phase - app. 16 days

• Desk review conducted
• Briefings of evaluator
• An inception report will be prepared by the evaluator detailing the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods, proposed sources of data and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables. 6 days

Stakeholder consultations and Interviews - app. 4 days

• The evaluator will consult with all relevant stakeholders and conduct a series of interviews, focus group discussions, and field visits in order to collect the required data. 

Analysis of data and drafting report - app. 8 days

• Once the data is collected, the evaluator will analyse them and draft the evaluation report. 

Presentation of draft evaluation report to Stakeholders and in the steering committee meeting - app. 6 days

• Once the draft final evaluation report submitted, it will be presented to all stakeholder and steering committee for reviewing. The comments shared by the stakeholder and steering committee will be incorporated into the final evaluation report. 

Final Report - app. 4 days

• The evaluator will revise the final evaluation report based on the comments and inputs provided by all stakeholders and submit the final report to UNDP. 

Total number of working days 40 days

Location:
Home-based with possible missions to Serbia and Moldova

Competencies

Corporate competencies: 

• Demonstrates integrity by modeling the UN’s values and ethical standards;
• Promotes the vision, mission, and strategic goals of UNDP; 
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability; 
• Treats all people fairly without favoritism; 
• Fulfills all obligations to gender sensitivity and zero tolerance for sexual harassment. 

Functional competencies:

• Strong interpersonal and managerial skills;
• Ability to work with people from different backgrounds 
• Evidence of delivering good quality evaluation and research products in a timely manner is required;
• Good writing and reporting skills;
• Good presentation skills;
• Ability to work under pressure and stressful situations, and to meet tight deadlines.

Required Skills and Experience

The Individual consultant should have the following skills/competencies and characteristics:

Education:

• At least bachelor’s degree in Public Policy and Management, Public Administration, Development studies, International Development, Finance or a similar field is required;

Experience:

• At least 7 years accumulated experience in project/programme evaluation is required;
• At least 5 years accumulated experience in programme management support, programme/project formulation, monitoring and evaluation and RBM implementation is an asset;
• Knowledge and experience in the field of innovation, cities initiatives, alternative finance mechanisms is an asset;

Language skills:

• Excellent written and verbal communication skills in English is required.

Evaluation of Applicants

Individual consultants will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal. The award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:  responsive/compliant/acceptable, and having received the highest score out of a pre-determined set of weighted technical (P11 desk reviews, interviews and methodology evaluation) and financial criteria specific to the solicitation. 

Only candidates who receive min 70% of points in desk review (Criteria A-E) will be considered for interviews. Only candidates who receive min 70% of points in technical evaluation (Criteria A-G) will be considered for the financial evaluation.

Technical Criteria - 70% of total evaluation – max. 35 points:

• Criteria A - bachelor’s degree in Public Policy and Management, Public Administration, Development studies, International Development, Finance or a similar field– max points: 8;
• Criteria B – 7 years accumulated experience in project/programme evaluation - max points: 5;
• Criteria C – 5 years accumulated experience in programme management support, programme/project formulation, monitoring and evaluation and RBM implementation - max points: 5;
• Criteria D - Knowledge and experience in the field of innovation, cities initiatives, alternative finance mechanisms- max points: 5;
• Criteria E - Methodology - max points: 4;
• Criteria F - Interview Criteria - Strong interpersonal and managerial skills, ability to work with people from different backgrounds and evidence of delivering good quality evaluation and research products in a timely manner - max points: 4;
• Criteria G – Interview Criteria - verbal communication skills in English - max points: 4;

Financial Criteria - 30% of total evaluation – max. 15 points.

Application Procedures 

The application submission is a two-step process. Failing to comply with the submission process may result in disqualifying the applications.

Step 1: Interested candidates must include the following documents when submitting the applications (Please group all your documents into one (1) single PDF attachment as the system only allows upload of one document):

• Cover letter explaining why you are the most suitable candidate for the advertised position and a brief methodology on how you will approach and conduct the work. Please paste the letter into the "Resume and Motivation" section of the electronic application. 
• Filled P11 form or CV including past experience in similar projects and contact details of referees  (blank form can be downloaded fromhttp://www.eurasia.undp.org/content/dam/rbec/docs/P11_modified_for_SCs_and_ICs.do); 

Step 2: Submission of Financial Proposal – Only shortlisted candidates will be contacted and requested to provide a financial offer.

Payments will be made only upon confirmation of UNDP on delivering on the contract obligations in a satisfactory manner. 

Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director. Consultants are also required to comply with the UN security directives set forth under dss.un.org

General Terms and conditions as well as other related documents can be found under: http://on.undp.org/t7fJs.

Qualified women and members of minorities are encouraged to apply. 

Due to large number of applications we receive, we are able to inform only the successful candidates about the outcome or status of the selection process.