Background

INTRODUCTION

In accordance with UNDP and GEF M&E policies and procedures, all full and medium-sized UNDP support GEF financed projects are required to undergo a terminal evaluation upon completion of implementation. These terms of reference (TOR) sets out the expectations for a Terminal Evaluation (TE) of the full sized project entitled “Scaling Up Implementation of the Sustainable Development Strategy for the Seas of East Asia” (PIMS 4752) implemented through the Partnership for the Environmental Management for the Seas of East Asia (PEMSEA). The Project started in August 2014 and is in its last year of implementation.

Project Title 

SCALING UP IMPLEMENTATION OF THE SUSTAINABLE DEVELOPMENT STRATEGY FOR THE SEAS OF EAST ASIA

Project Description

The UNDP/GEF Project on Scaling up the Implementation of the Sustainable Development Strategy for the Seas of East Asia (SDS-SEA) is a GEF project being implemented by UNDP with UNDP Philippines serving as Principal Project Representative (PPR) and the Partnerships in Environmental Management for the Seas of East Asia (PEMSEA) Resource Facility (PRF) as the implementing partner in accordance to the agreement entered into by the two parties. Countries bordering the East Asian Region - Cambodia, China, Indonesia, Lao PDR, Philippines, Thailand, Timor Leste and Vietnam are the eight (8) participating countries to the project while Japan, Republic of Korea and Singapore provide co-financing to the Project.

The Project is the fourth phase of the UNDP-GEF projects under PEMSEA[1]. It represents the “transformation phase” of GEF support culminating in the sustainability of PEMSEA as the regional coordinating mechanism for the implementation of SDS-SEA, and makes a stronger linkage between sustainable development of river basins, coastal and marine areas and local, national and regional investment processes in a blue economy.

The Project is consistent with International Waters Objectives 2 and 3 in the GEF 5 Focal Area Strategies which are to: 1) Catalyze multi-state cooperation to rebuild marine fisheries and reduce pollution of coasts and large marine ecosystems while considering climatic variability and change, and 2) Support foundational capacity building, portfolio learning, and targeted research needs for ecosystem-based joint management of trans-boundary water systems.

[1] Pilot phase project (1994-1999): “Marine Pollution Prevention and Management of the East Asian Seas Region.”; Second phase project (1999-2008): “Building Partnerships in Environmental Management for the Seas of East Asia”; Third phase project (2008-2013): “Implementation of the Sustainable Development Strategy for the Seas of East Asia”.

The Mid-Term Review, which reviewed the status of the Project in 2014 through to March 2018 recommended a 12-month extension to allow sufficient time to achieve progress towards the outcomes in countries that have been delayed in starting implementation due to some administrative requirements. The 12-month extension was endorsed by the 8 participating countries at the 2019 PSC meeting, including the adoption of revised indicators and end of project targets that were incorporated into the Strategic Results Framework of the Project Document. PRF received the formal notification from UNDP on 18 June 2019, extending the project to 31 August 2020. Due to the COVID19 pandemic, the project was granted a second extension until 31 December 2020.

Institutional Arrangements

The principal responsibility for managing the TE resides with the UNDP Philippines Country Office (UNDP PHL CO) under the Programme Team Leader of the Climate Action Programme Team.  All reports shall be reviewed and endorsed by the Team Leader of the Climate Action Team and the Regional Technical Advisor in Bangkok Regional Hub.

PEMSEA, as the Implementing Partner will be responsible for liaising with the TE consultant to provide all relevant documents and set up schedules for virtual stakeholder interviews.

Duration of the Work

To undertake the tasks and deliver the expected outputs, the consultant shall be engaged for a period of 40 working days spread over 10 weeks. Effectivity of the engagement will be upon signing of the contract and will be valid until all outputs have been delivered and accepted. The target start-of-work date is 15 October 2020 and the expected completion date is 31 December 2020.

Duty Station

The position is home-based and for accessibility and availability to allow for discussions and reporting on progress of activities regular online meetings shall be conducted.

In light of the COVID-19 pandemic all work of the Individual consultant shall be done within the guidelines and protocols set by their respective governments. This engagement does not have any travel-related activities,

The Consultant will not be required to report physically but status reports on the outputs shall be expected from time to time.

Qualifications of the Successful Individual Contractor

An international consultant shall be hired as the evaluator to prepare the Terminal Evaluation Report and other outputs as specified in the TOR.

The International Consultant should have prior experience in evaluating similar projects.    The evaluators selected should not have participated in the project preparation and/or implementation and should not have a conflict of interest with project-related activities.

Evaluator Ethics

The consultant shall hold to the highest ethical standards and is required to sign a code of conduct upon acceptance of the assignment. This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The evaluator must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing the collection of data and reporting on data. The evaluator must also ensure the security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.

Scope of Price Proposal and Schedule of Payments

This is a fixed output-based contract price regardless of extension of the herein specific duration. The consultant will be paid an all-inclusive lump sum amount (i.e. professional fees, communications including internet). 

Payment schedule

Duties and Responsibilities

Expected Outputs and Deliverables

The objective of the Terminal Evaluation is to enable the GEF, UNDP and the participating countries to assess the relevance, efficiency, effectiveness, impact and sustainability of the Scaling Up Implementation of the Sustainable Development Strategy for the Seas of East Asia Project. The Terminal Evaluation will assess achievements of the project against its objectives.  It will also identify factors that have facilitated or impeded the achievement of the objectives. While a thorough review of the past is in itself very important, the in-depth evaluation is expected to lead to detailed overview and lessons learned for the future and particularly provide recommendations that will contribute to sustaining the outcomes of the project to the stakeholders including PEMSEA as a regional mechanism for SDS SEA implementation.

The Terminal Evaluation will assess project performance against expectations set out in the project’s Logical Framework/Results Framework. The Terminal Evaluation will assess results according to the criteria outlined in the Guidance for Terminal Evaluation of UNDP-supported GEF-financed Projects. The Independent Evaluation Office (IEO) of UNDP has updated the COVID-19 evaluation guidance, issued in April 2020, supporting evaluation planning and implementation during COVID-19.

As part of initial deliverables of the consultant, an Inception Report will be prepared for discussion with UNDP and PEMSEA. This will outline the proposed approach to the assignment and will include, but not be limited to, a detailed work plan of activities, and methodologies of approach. It is anticipated that the Consultant will look at the entire evaluation and its activities in a holistic manner to maximizes efficiencies.

The Inception Report should be produced before the virtual interviews are undertaken to ensure that methods are aligned with the GEF guidelines for final evaluation.

An overall approach and method[1] for conducting project terminal evaluations of UNDP supported GEF financed projects has been adjusted to consider the COVID19 pandemic, particularly on the conduct of field visits. The evaluation should include a mixed methodology of document review, virtual interviews, and discussion with country evaluators. The consultant must conduct the following:

1. Desk review of project documents, outputs, monitoring reports, mid-term evaluation report,
2. Review of specific products including datasets, management and action plans, publications and other material and reports;
3. Interviews with the UNDP Regional Technical Advisor from the Bangkok Regional Hub, and UNDP CO in the Philippines, as the Implementing Agency of the project
4. Interviews with the PEMSEA Executive Director, Project Manager and other project staff; and
5. Consultations and/or interviews with relevant stakeholders involved, including government representatives from the participating countries
6. National In-country evaluators shall be hired by the PEMSEA Resource Facility to prepare a more in depth evaluation of the in country activities and achievements and shall provide the reports to the Terminal Evaluator Consultant.

[1] For additional information on methods, see the Handbook on Planning, Monitoring and Evaluating for Development Results, Chapter 7, pg. 163

The Full Terminal Evaluation Report shall include, at the minimum the following evaluation parameters:

Project Design/Formulation

• Theory of Change
• Gender equality and women’s empowerment
• Social and Environmental Safeguards
• Analysis of Results Framework: project logic and strategy, indicators
• Assumptions and Risks

• Lessons from other relevant projects (e.g. same focal area) incorporated into project design
• Planned stakeholder participation
• Linkages between project and other interventions within the sector
• Management arrangements

Project Implementation

• Adaptive management (changes to the project design and project outputs during implementation)
• Actual stakeholder participation and partnership arrangements
• Project Finance and Co-finance
• Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*)
• Implementing Agency (UNDP) (*) and Executing Agency (*), overall project oversight/implementation and execution (*)
• Risk Management, including Social and Environmental Standards.

Project Monitoring and Evaluation. The evaluation will assess whether the project met the minimum requirements for project design of M&E and the implementation of the project M&E plan.

Projects should have a sound M&E plan to monitor results and track progress toward achieving project objectives. An M&E plan should include a baseline (including data, methodology, and so on), SMART (specific, measurable, achievable, realistic, and timely) indicators and data analysis systems, and evaluation studies at specific times to assess results and adequate funding for M&E activities. The time frame for various M&E activities and standards for outputs should have been specified.

The evaluation should verify that an M&E system was in place and facilitated timely tracking of progress toward project objectives by collecting information on chosen indicators continually throughout the project implementation period; annual project reports were complete and accurate, with well-justified ratings; the information provided by the M&E system was used during the project to improve performance and to adapt to changing needs; and projects had an M&E system in place with proper training for parties responsible for M&E activities to ensure that data will continue to be collected and used after project closure.

Budgeting and funding for M&E activities. In addition to incorporating information on funding for M&E while assessing M&E design, the evaluator will determine whether M&E was sufficiently budgeted for at the project planning stage and whether M&E was funded adequately and in a timely manner during implementation

Project M&E systems will be rated as follows on quality of M&E design and quality of M&E implementation:

1. Highly satisfactory (HS). There were no shortcomings in the project M&E system.
2. Satisfactory (S). There were minor shortcomings in the project M&E system.
3. Moderately satisfactory (MS). There were moderate shortcomings in the project M&E system.
4. Moderately unsatisfactory (MU). There were significant shortcomings in the project M&E system.
5. Unsatisfactory (U). There were major shortcomings in the project M&E system.

Project Co-financing. The Evaluation will assess the key financial aspects of the project, including the extent of co-financing planned and realized. Project cost and funding data will be required, including annual expenditures.  Variances between planned and actual expenditures will need to be assessed and explained.  Results from recent financial audits, as available, should be taken into consideration. The evaluator(s) will receive assistance from the Country Office (CO) and Project Team to obtain financial data in order to complete the co-financing table below, which will be included in the terminal evaluation report.

Project Results

• Assess the achievement of outcomes against indicators by reporting on the level of progress for each objective and outcome indicator at the time of the TE and noting final achievements
• Relevance (*), Effectiveness (*), Efficiency (*) and overall project outcome (*)
• Sustainability: financial (*), socio-political (*), institutional framework and governance (*), environmental (*), overall likelihood of sustainability (*)
• Country ownership
• Gender equality and women’s empowerment
• Cross-cutting issues (poverty alleviation, improved governance, climate change mitigation and adaptation, disaster prevention and recovery, human rights, capacity development, South-South cooperation, knowledge management, volunteerism, etc., as relevant)
• GEF Additionality
• Catalytic Role / Replication Effect
• Progress to impact

The evaluator is expected to frame the evaluation effort using the criteria of relevance, effectiveness, efficiency, sustainability, and impact, as defined and explained in the UNDP Guidance for Conducting Terminal Evaluations of UNDP-supported, GEF-financed Projects.  The evaluation must provide evidence-based information that is credible, reliable and useful.

The evaluation will assess the achievement of outputs and outcomes and provide ratings for targeted objectives and outcomes. The assessment of project results seeks to determine the extent to which the project objectives were achieved, or are expected to be achieved, and determine if the project has led to any other short- or long-term and positive or negative consequences. In assessing project results, the Consultant will seek to determine the extent of achievement and shortcomings in reaching project objectives as stated in the project appraisal document and indicate if there were any changes and whether those changes were approved. In assessing project performance, the Consultant can focus on achievements in terms of outputs, outcomes, or impacts. Although the GEF is more interested in assessing impacts, these may take a long time to manifest. On the other end, output achievement is easy to assess but tells very little about whether GEF investments were effective in delivering global environmental benefits. Focus on outcomes is, therefore, an appropriate compromise.  It captures project efficacy in terms of delivering medium-term expected results, thus assessment of project outcomes should be a priority. 

To assess level of achievement of outcomes and objectives, these three criteria will be used in the evaluation to assess level of achievement of project outcomes and objectives.

• Relevance. Were the project’s outcomes consistent with the focal areas/operational program strategies and country priorities?
• Effectiveness. Are the actual project outcomes commensurate with the original or modified project objectives? If the original or modified expected results are merely outputs/inputs, the evaluators should assess if there were any real outcomes of the project and, if there were, determine whether these are commensurate with realistic expectations from such projects,

Efficiency. Was the project cost effective? Was the project the least cost option? Was project implementation delayed, and, if it was, did that affect cost effectiveness? Wherever possible, the evaluator should also compare the costs incurred and the time taken to achieve outcomes with that for similar projects

The evaluation of relevance, effectiveness, and efficiency will be as objective as possible and will include sufficient and convincing empirical evidence. Ideally, the project monitoring system should deliver quantifiable information that can lead to a robust assessment of project effectiveness and efficiency. Since projects have different objectives, assessed results are not comparable and cannot be aggregated. Outcomes will be rated as follows for relevance, effectiveness, and efficiency:

1. Highly satisfactory (HS). The project had no shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.
2. Satisfactory (S). The project had minor shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.
3. Moderately satisfactory (MS). The project had moderate shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.
4. Moderately unsatisfactory (MU). The project had significant shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.
5. Unsatisfactory (U). The project had major shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.
6. Highly unsatisfactory (HU). The project had severe shortcomings in the achievement of its objectives in terms of relevance, effectiveness, or efficiency.

Sustainability. Sustainability is understood as the likelihood of continued benefits after the GEF project ends. Given the uncertainties involved, it may be difficult to have a realistic a priori assessment of sustainability of outcomes. Therefore, assessment of sustainability of outcomes will give special attention to analysis of the risks that are likely to affect the persistence of project outcomes. This assessment should explain how the risks to project outcomes will affect continuation of benefits after the GEF project ends. It will include both exogenous and endogenous risks. For sustainability of outcomes the following ratings should be provided:

1. Likely (L). There are no or negligible risks that affect this dimension of sustainability.
2. Moderately likely (ML). There are moderate risks that affect this dimension of sustainability.
3. Moderately unlikely (MU). There are significant risks that affect this dimension of sustainability.
4. Unlikely (U). There are severe risks that affect this dimension of sustainability.

Main Findings, Conclusions, Recommendations and Lessons Learned

• The TE will include a summary of the main findings of the TE report. Findings should be presented as statements of fact that are based on analysis of the data.

• In-country assessment report of the 8 participating countries will be attached to the TE report and major findings to be included in the summary.
• The section on conclusions will be written in light of the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the TE findings. They should highlight the strengths, weaknesses and results of the project, respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries, UNDP and the GEF, including issues in relation to gender equality and women’s empowerment.
• Recommendations should provide concrete, practical, feasible, and targeted recommendations directed to the intended users of the evaluation about what actions to take and decisions to make. The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation.
• The TE report should also include lessons that can be taken from the evaluation, including best practices in addressing issues relating to relevance, performance and success that can provide knowledge gained from the particular circumstance (programmatic and evaluation methods used, partnerships, financial leveraging, etc.) that are applicable to other GEF and UNDP interventions. When possible, the TE team should include examples of good practices in project design and implementation.
• It is important for the conclusions, recommendations and lessons learned of the TE report to include results related to gender equality and empowerment of women.

*The final TE report must be in English.

**Audit Trail - details how all received comments have (and have not) been addressed in the final TE report, to the Commissioning Unit

All final TE reports will be quality assessed by the UNDP Independent Evaluation Office (IEO).  Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines.[1]

[1] Access at: http://web.undp.org/evaluation/guideline/section-6.shtml

Competencies

Corporate competencies

• Demonstrates integrity by modeling the UN's values and ethical standards;
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality, and age sensitivity and adaptability;
• Treats all people fairly without favoritism.

Functional and technical competencies

• Ability to work in a diverse and multi-cultural environment;
• Self-motivated and ability to work under pressure and to meet strict and competing deadlines;
• Displays analytical judgment and demonstrated ability to handle confidential and politically sensitive issues in a responsible and mature manner;
• Demonstrates openness to change and ability to manage complexities;

Required Skills and Experience

Candidates will be evaluated based on the Combined Rating methodology:

• Technical qualifications = 70%
• Financial Proposal =    30%

Applicants who will only receive 70 points from the assessment of the CV will be qualified for the assessment of the Financial Proposal. 

Recommended Presentation of Offer

Interested applicants must submit the following documents/information to demonstrate their qualifications. Please group them into one (1) single PDF document as the application only allows to upload maximum one document. 

1. Personal CV or P11, indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references; and
2. Duly accomplished Letter of Confirmation of Interest and Availability using the template provided by UNDP; ?Financial Proposal that indicates the all-inclusive fixed total contract price, supported by a breakdown of costs, as per template provided.  If an Offeror is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the Offeror must indicate at this point and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.
3. Curriculum vitae (CV), indicating all past experiences from similar projects, to prove criteria cited in technical qualification. The applicant must indicate specifically his/her role in the aforementioned experiences, e.g, Team leader, team member, Lead evaluator, etc. below is a proposed format to be provided, as well as the contact details (email and telephone number) of the candidate and at least three (3) professional references;

4.  

   Activity Conducted/ Project Name	Date Completed	Role and Responsibilities  in Activity	Client for whom output is intended
   e.g Evaluation of IW project  x	Dec. 2016	Team Lead	UNDP Samoa

Interested applicants to note that personal Medical/health insurance (to be purchased by the individual at his/her own expense) is mandatory for the issuance of contracts. Upon award of the contract, the consultant must be ready to submit proof of insurance valid during the contract duration.

The following templates / Annexes and IC General Terms & Conditions can be downloaded from http://gofile.me/6xdJm/bE9TCw8fU:

• General Terms and Conditions for Individual Contract
• Letter of Confirmation of Interest and Availability
• P-11 form

In view of the volume of applications, UNDP receives, only shortlisted offerors will be notified.