Antecedentes
This is the Terms of Reference (ToR) for the Midterm Review (MTR) of the ‘EU4Climate’ project implemented by UNDP and funded by EU; Project number 00115652; implemented through the UNDP Direct Implementation Modality in the six EU Eastern Partnership countries: Armenia, Azerbaijan, Belarus, Georgia, Moldova, Ukraine[1]. The project’s budget is EUR 8,800,000; including EUR 8,000,000 contribution by the EU and EUR 800,000 cofinance by UNDP. The MTR is to be undertaken over a three months period in 2021. The project started on 14 December 2018 and is in its third year of implementation. This ToR sets out the expectations for this MTR.
The EU4Climate Project helps governments in the six EU Eastern Partner countries - Armenia, Azerbaijan, Belarus, Georgia, the Republic of Moldova and Ukraine - to take action against climate change. It supports countries in implementing the Paris Climate Agreement and improving climate policies and legislation. Its ambition is to limit climate change impact on citizens lives and make them more resilient to it. EU4Climate is funded by the European Union (EU) and implemented by the United Nations Development Programme (UNDP).
The objective of the project is to support the development and implementation of climate-related policies by the Eastern Partnership countries that contribute to their low emission and climate resilient development and their commitments to the Paris Agreement on Climate Change. It identifies key actions and results in line with the Paris Agreement, the "20 Deliverables for 2020”, and the key global policy goals set by the UN 2030 Agenda for Sustainable Development. The project will also translate into action priorities outlined in the Eastern Partnership Ministerial Declaration on Environment and Climate Change of October 2016.
The following results are expected to be achieved by the project: (i) Finalized/up-dated nationally determined contributions and national mid-century strategies and communicated to the United Nations Framework Convention on Climate Change (UNFCCC), (ii) Improved inter-institutional awareness and coordination at political and technical level of the Paris Agreement and the corresponding national commitments, (iii) Established or strengthened measurement, reporting and verification (MRV) systems, with countries getting on track with Paris Agreement transparency requirements, (iv) Establishment of concrete sectoral guidelines for the implementation of the Paris Agreement in each of the Eastern Partners, especially in the field of energy (v) Advanced alignment with EU acquis as provided by bilateral agreements with EU and in the context of the Energy Community Treaty, (vi) Increased mobilization of climate finance, and (vii) Enhanced adaptation planning. The project was designed to operate on both regional and country level; the fact that it was designed to promote ownership and promote learning, knowledge, dialogue with a view to maximise capacity building effect; importance of coordination and synergies with a view to leverage partnerships. See also the project’s logical framework in Annex 1. A result-oriented monitoring (ROM) has been conducted for EU4Climate in 2020; the results of ROM are to be taken into account during the MTR.
[1] A detailed description of the project and its key stakeholders is provided in the project’s Description of the Action, and will be provided to the Consultant upon signing the service agreement with UNDP
Annexes can be found under the link: https://procurement-notices.undp.org/view_notice.cfm?notice_id=81278
Deberes y responsabilidades
EVALUATION PURPOSE, SCOPE AND OBJECTIVES
The main objective of the MTR is to assess progress towards the achievement of the project objectives and outcomes as specified in the Project Document and assess signs of project success or failure with the goal of identifying the necessary changes to be made in order to set the project on-track to achieve its intended results, taking into account problems and opportunities. Virtual visits are expected to be undertaken by the consultant to all six countries participating in the project (Armenia, Azerbaijan, Belarus, Georgia, Moldova, Ukraine). The virtual visits shall include videoconference discussion with the project’s key stakeholders: representatives of the focal ministries in each of the countries, EU Delegation representatives, project staff.
The scope of the MTR includes the entirety of EU4Climate activities covering from 2019 to date. By reviewing the Logical Framework indicators against progress made towards the project outputs targets, using a Results Matrix with color code progress in a “traffic light system” based on the level of progress achieved, the MTR consultant assigns a rating on progress for the project objective and each outcome and makes recommendations from the areas marked as “not on target to be achieved” (red)[1]. The MTR will also examine the contribution of EU4Climate toward cross-cutting issues, e.g., gender equality and capacity development of the host countries’ governments. The MTR should be forward-looking by drawing lessons from the last years’ project implementation and propose recommendations for the coming years.
[1] Evaluation matrix sample is provided under Annex 3
Annexes can be found under the link: https://procurement-notices.undp.org/view_notice.cfm?notice_id=81278
EVALUATION CRITERIA AND KEY GUIDING QUESTIONS
The MTR will answer these broad questions as follows:
1) What did EU4Climate intend to achieve during the period under review?
2) To what extent has the project achieved (or is likely to achieve) its intended objectives at the output level, and what contribution has it made at the outcome level?
3) What factors contributed to or hindered the project’s performance and eventually, the sustainability of results?
4) What needs to be done in the remaining project lifetime to ensure achievement of the objectives, which were not achieved or were partially achieved during the review period.
In addition to the above questions, the MTR is expected to produce answers surrounding the evaluation criteria of relevance, effectiveness, efficiency and sustainability. Below are guiding questions and areas for review:
Relevance
• To what extent has the project responded to the priorities and the needs of target beneficiaries as defined in the project document?
• Has the project been able to effectively adapt its areas of work to the effects of the COVID-19 pandemic in projects’ implementation countries?
• Review the Theory of Change of the project if relevant.
• Review how the project addresses country priorities and if it is aligned with the national development, and UNDP Regional Programme for Europe and the CIS.
• Review the functionality of project governance structure, i.e. the steering committee.
• Undertake a critical analysis of the project’s Logical framework indicators and targets, baseline data, assess how “SMART” the midterm and end-of-project targets are (Specific, Measurable, Attainable, Relevant, Timebound), and suggest specific amendments/revisions to the targets and indicators as necessary.
Effectiveness
• By reviewing the results and resources framework, is the project on track to achieve intended results at the outcome and output levels? What are the key achievements and what factors contributed to the achievements or non-achievement of those results?
• In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
• In which areas does the project have the least achievements? What have been the constraining factors and why? How can they or could they be overcome?
• To what extent has the project been appropriately responsive to the needs of the national constituents and changing partner priorities?
- How the project has contributed to the partner governments’ relevant policies / actions?
• Identify challenges encountered and remaining barriers to achieving the project objective.
• By reviewing the aspects of the project that have already been successful, identify ways in which the project can further expand these benefits.
• Has the project been effective in addressing the impact of the COVID-19 pandemic, both in terms of effective implementation of the planned actions, and in assisting the partner governments with readiness to post-COVID recovery?
Efficiency
• To what extent is the project management structure as outlined in the Project Document efficient in generating the expected results?
• Examine how the COVID 19 pandemics has contributed/could further contribute to additional delays and the risk of not achieving the project objectives and targets and propose measures to adapt to the situation.
• Assess whether the combined expertise of the project team is adequate to deliver against the project objectives and targets.
• Review any delays in project start-up and implementation, identify the causes and examine if they have been resolved.
• Review the changes to fund allocations as a result of budget revisions and assess the appropriateness and relevance of such revisions. Examine possible funding shortfalls and their likely impact.
• To what extent has there been an economical use of financial and human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
• Does the project have the appropriate financial controls, including reporting and planning, that allow management to make informed decisions regarding the budget and allow for timely flow of funds?
Sustainability
• What outcomes and outputs have the most likelihood of sustainability and being adopted by partners and why?
• To what extent do national partners have the institutional capacities, including sustainability strategies, in place to sustain the outcome-level results?
• To what extent have national partners committed to providing continuing support (financial, staff, aspirational, etc.)?
• To what extent do partnerships exist with other national institutions, NGOs, United Nations agencies, the private sector and development partners to sustain the attained results?
• What is the possible impact of Covid-19 on project’s sustainability?
Visibility
• Review internal project communication with stakeholders: Is communication regular and effective? What feedback mechanisms are in place?
• Review external project communication: Are proper means of communication established or being established to express the project progress and intended impact to the public (is there a web presence? Did the project implement appropriate communication tools?) and ensuring donors’ visibility.
Gender equality:
• To what extent has gender equality and the empowerment of women been addressed in the design, implementation, monitoring and reporting of the project?
• To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?
METHODOLOGY
The MTR methodology will adhere to the United Nations Evaluation Group (UNEG) Norms & Standards. The MTR will be carried out by an independent consultant who will adopt an integrated approach involving a combination of data collection and analysis tools to generate concrete evidence to substantiate all findings. Evidence obtained and used to assess the results of project’s support should be triangulated from a variety of sources, including verifiable data on indicator achievement, existing reports, evaluations and technical papers, stakeholder interviews, surveys and site visits where/when possible. It is expected that the evaluation methodology will comprise of the following elements:
• Review documents (Desk Review): the MTR consultant will conduct a desk review of all relevant sources of information i.e., the Project Document, progress reports, inception report, M&E Framework, roles and responsibilities, management arrangements, project budget revisions, internal M&E data, results of the Result Oriented Monitoring (ROM), legal documents and any other materials that the EU4Climate team considers useful for the evidence-based review.
• Interview with key stakeholders including videoconference meetings, online surveys interview et al, ensuring close engagement with the project’s Steering Committee members (EC Directorate General for Neighborhood and Enlargement; EC Directorate General for Climate Action; Ministry of Environment of Armenia; Ministry of Ecology and Natural Resources of Azerbaijan; Ministry of Natural Resources and Environmental Protection of Belarus; Ministry of Environmental Protection and Agriculture of Georgia; Ministry of Agriculture, Regional Development and Environment of Moldova; Ministry of Environmental Protection and Natural Resources of Ukraine); implementing partners (the Energy Community Secretariat, Environment Agency Austria), senior officials and national project coordinators, key experts and consultants in the subject area, project stakeholders, academia, CSOs, etc.
• Consultations with beneficiaries through interviews and/ or focus group discussions;
• Survey and/ or questionnaires where appropriate;
• Triangulation of information collected from different sources/methods to enhance the validity of the findings.
The evaluation is expected to use a variety of data sources, primary, secondary, qualitative, quantitative, etc. to be extracted through surveys, storytelling, focus group discussions, face to face interviews, participatory methods, desk reviews, etc. conducted with a variety of partners. A transparent and participatory multi stakeholder approach should be followed for data collection from government partners, civil society, private sector etc. Evidence will be provided for every claim generated by the evaluation and data will be triangulated to ensure validity. An evaluation matrix or other methods can be used to map the data and triangulate the available evidence.
Special note: Given the ongoing COVID 19 pandemic and the resultant restrictions may require many of the in-person missions/consultations and data gathering / activities to be carried out remotely using videoconferencing means.
In addition to reviewing the documents relating to EU4Climate project, the consultant should visit UNDP Independence Evaluation Office’s website http://web.undp.org/evaluation/guideline/index.shtml to be updated with UNDP's relevant information and documents required.
Duties and responsibilities
Time frame for the evaluation process
The total duration of the MTR will be approximately 40 days over a period of three months with an estimated start date of 15 September 2021. Of this total of 40 days, a minimum of 10 working days, not including weekends, should be spent by the international consultant in teleconference meetings with the project stakeholders.
Exact deadlines for each activity of MTR will be determined at the time of contract issuance. The tentative MTR timeframe is as follows (estimated total number of days - 40):
• Timeframe: 15 August 2021; Activity: Application closes;
• Timeframe: 25 August 2021; Activity: Select and contract MTR Consultant;
• Timeframe: 1 September 2021; Activity: Distribution of all documents and reports to the MTR Consultant;
• Timeframe: September 2021 (4 working days); Activity: MTR Inception report and workplan prepared; debriefing to UNDP and key stakeholders regarding the inception report.
• Timeframe: 10 working days in September-October 2021; Activity: Videoconference interviews with the project stakeholders; shall be a minimum of 10 working days, not including weekends;
• Timeframe: 10 October 2021; Activity: Mission wrap-up meeting. Debriefing to UNDP summarizing with initial findings at the end of the MTR mission;
• Timeframe: Before 15 November 2021 (22 working days); Activity: Preparation and submission of the draft report by the international consultant;
• Timeframe: November 2021; Activity: Incorporating audit trail from feedback on draft report; Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
• Timeframe: Before the end of November 2021; Activity: Preparation & Issue of Management Response; UNDP IRH is responsible for the management response.
• Timeframe: Before the end of November 2021 (4 working days); Activity: Finalization of MTR report. Expected date of full MTR completion;
Activities:
• Activity: Preparation to the MTR: documents review and preparing MTR Inception Report; Tentative Timeframe: During the first week after signing a contract;
• Activity: 10 working days - stakeholder meetings in videoconference mode. Tentative Timeframe: Within three weeks of the commencement of the work (September 2021);
• Activity: Mission wrap-up meeting & preparation of initial findings; Tentative Timeframe: End of MTR videoconference mission (before the end of October 2021);
• Activity: Submission of the draft report; Tentative Timeframe: Within four weeks after end of MTR virtual mission, expected to be by the end of October 2021;
• Activity: Final Report; Tentative Timeframe: Within two weeks after receiving feedback from the counterparts on the draft report, expected to be by the end of November 2021.
The list of proposed stakeholders to interview should be provided in the Inception Report.
Reporting and delivery of outputs
Implementation arrangements
The principal responsibility for managing this MTR resides with the EU4Climate Project Manager. The Project Team will be responsible for liaising with the MTR consultant to provide all relevant documents and set up stakeholder interviews.
The MTR is to be performed by an independent international consultant with experience and exposure to projects and evaluations in other regions globally will lead the MTR. The international consultant will be supported by project team. The international consultant cannot have participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) and should not have a conflict of interest with project’s related activities.
Evaluation products (deliverables)
• Deliverable 1: MTR Inception Report. Description: the inception report will detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of scope of the work and intended work plan of the analysis, proposed methodology and evaluation questions, proposed schedule of tasks, proposed data sources and data collection procedures, activities and deliverables. MTR consultant clarifies objectives and methods of Midterm Review; Timing: by 30 September 2021
• Deliverable 2: Draft Final Report. Description: Full report with annexes (see Annex 5. UNDP evaluation report template and quality standards; Timing: 30 October 2021;
• Deliverable 3: Final Report; Description: Revised report with audit trail detailing how all received comments have (and have not) been addressed in the final MTR report; Timing: 30November 2021
Payments:
The international consultant will be paid in 3 instalments as follows:
• 10% of payment upon submission and acceptance by the Project Manager of the Deliverable 1. MTR Inception Report;
• 30% upon submission and acceptance by the Project Manager of the Deliverable 2. Draft MTR report
• 60% upon submission and acceptance by the Project Manager of the Deliverable 3. Final MTR report
Timing and travel: The Consultant will be engaged under the Individual Contract. The engagement will be app. 40 working days.
This is a home-based assignment without travel envisaged. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and International Consultant, prior to travel and will be reimbursed. In general, UNDP should not accept travel costs exceeding those of an economy class ticket. Should the IC wish to travel on a higher class he/she should do so using their own resources. Approval of the IRH CDT Team Leader is required prior to planning of the trips and relevant logistics.
Reporting language:
Deliverables will be delivered in English.
EVALUATION ETHICS
This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.
Competencias
Corporate Competencies:
• Demonstrates integrity by modeling the UN’s values and ethical standards;
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability;
• Treats all people fairly without favoritism;
• Fulfills all obligations to gender sensitivity and zero tolerance for sexual harassment.
Functional Competencies:
• Competence in adaptive management;
• Knowledge of and work experience in the energy efficiency related water and agriculture projects, including those funded by the EU;
• Excellent training, facilitation and communication skills;
• Results driven, ability to work under pressure and to meet required deadlines;
• Good understanding and experience in the field of climate change policies.
Habilidades y experiencia requeridas
Education:
• Master’s degree in Energy, Environment, Business Administration, Economics, Engineering or related field;
Experience:
• At least 10-year work experience and proven track record with policy advice and/or project development/implementation in climate change or energy efficiency in the developing/transition economies;
• Experience working with at least two project evaluations, including experience with SMART based indicators (Project evaluation/review experiences within United Nations system will be considered an asset);
• Experience working with international technical assistance projects in the EU Neighborhood countries region or EU accession candidate countries
Language requirements:
• English required; knowledge of Russian will be an asset.
- EVALUATION OF APPLICANTS
Individual consultant will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal.
The award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:
a) responsive/compliant/acceptable, and
b) Having received the highest score out of a pre-determined set of weighted technical (CV/P11 desk reviews and interviews) and financial criteria specific to the solicitation.
Only candidates who receive min 70% of points in desk review (Criteria A-E) will be considered for interviews. Only candidates who receive min 70% of points in technical evaluation (Criteria A-G) will be considered for the financial evaluation.
Technical Criteria - 70% of total evaluation – max. 70 points:
- Criteria A – (desk review) Advanced university degree in the fields related to Energy, Environment, Business Administration, Economics, Engineering – up to 5 points;
- Criteria B – (desk review) Experience working with the project evaluations within the past seven years including experience with SMART based indicators – up to 15 points;
- Criteria C – (desk review) Experience working with international technical assistance projects in the EU Neighborhood countries region – up to 10 points;
- Criteria D – (desk review) Methodology on the approach to conduct the work – up to 10 points;
- Criteria E – (desk review) At least two samples of the similar assignments delivered by the applicant – up to 10 points;
- Criteria F – (interviews) Experience working with the project evaluations – up to 20 points.
Financial Criteria - 30% of total evaluation – max. 30 points
Application procedure:
The application submission is a two-step process. Failing to comply with the submission process may result in disqualifying the applications.
Step 1: Interested candidates must include the following documents when submitting the applications (Please group all your documents into one (1) single PDF attachment as the system only allows upload of one document):
- Cover letter explaining why you are the most suitable candidate for the assignment
- Filled P11 form or CV including past experience in similar projects and contact details of referees (blank form can be downloaded from http://www.eurasia.undp.org/content/dam/rbec/docs/P11_modified_for_SCs_and_ICs.doc);
- Brief methodology on how you will approach and conduct the work
- At least two samples of the similar assignments delivered by the applicant. The samples shall be developed no earlier than August 2011.
- Financial Proposal* - specifying a total lump sum amount for the tasks specified in this announcement. The financial proposal shall include a breakdown of this lump sum amount (number of anticipated working days, travel, per diems and any other possible costs).
Payments will be made only upon confirmation of UNDP on delivering on the contract obligations in a satisfactory manner.
Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director. Consultants are also required to comply with the UN security directives set forth under dss.un.org
General Terms and conditions as well as other related documents can be found under: http://on.undp.org/t7fJs.
Qualified women and members of minorities are encouraged to apply.
Due to large number of applications we receive, we are able to inform only the successful candidates about the outcome or status of the selection process.
Annexes can be found under the link: https://procurement-notices.undp.org/view_notice.cfm?notice_id=81278