Background

1.Introduction

In accordance with UNDP and GEF M&E policies and procedures, all full- and medium-sized UNDP-supported GEF-financed projects are required to undergo a Terminal Evaluation (TE) at the end of the project.  This Terms of Reference (ToR) sets out the expectations for the TE of the medium sized project titled ‘PIMS 5281 Mainstreaming Biodiversity Conservation into River Management” implemented by the Department of Irrigation and Drainage (DID) Malaysia. The project started on 10 November 2016 and undergone an Independent Rapid Review (IRR) in 2019 to identify implementation issues and recommend corrective and adaptive measures to put the project back on track. Subsequent, a 12-month extension was approved on 21 August 2020 with new project operational closure date on 9 May 2022. Currently, the project is in its fifth year of implementation.  The TE process must follow the guidance outlined in the document ‘Guidance For Conducting Terminal Evaluations of UNDP-Supported, GEF-Financed Projects’ (http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEF-financedProjects.pdf).

2.Project Description 

Malaysia has some 157 river systems, as well a variety of tropical wetlands, forests and marine
ecosystems, representing several Global 200 Ecoregions, and it is recognized as one of 17 mega-diverse countries in the world. Its river systems as well as riparian and catchment forests support an immense diversity of aquatic and terrestrial biodiversity, including more than 600 freshwater fish species. These river systems provide ecosystem services benefiting both rural communities and urban societies, including water supply, artisanal fisheries, the aquarium fish industry, transport routes, tourism and recreation. However, Malaysia’s rivers face threats from a wide range of pressures that threaten their biodiversity and ecological stability, with ongoing loss of genetic resources, ecosystem services and national and local socio-economic benefits.

As called for in Malaysia’s Common Vision on Biodiversity, this project on “Mainstreaming Biodiversity Conservation into River Management” was developed to mainstream biodiversity conservation into riverine landscapes through improved river planning and management practices. This will be achieved through two interrelated components of work, to be undertaken in parallel:

Component 1: addresses the need for an operational national institutional framework and capacity for a more integrated and holistic approach to river management that takes riverine biodiversity into account;

Component 2: will demonstrate best management practices for riverine habitats in three different situations (a forested water supply reservoir catchment area, an urban river, and a rural river impacted by plantation development and smallholder land uses).

The global environmental benefits that will be secured by the overall project will result from strengthened sustainable management of Malaysia’s river systems and associated riverine buffer zones and catchment areas that specifically takes into account biodiversity conservation. Signed project document can be referred at https://www.my.undp.org/content/dam/malaysia/docs/EnE/EnEProDocs/Riverine%20Prodoc.pdf. Specifically, Key details of the project are as below:

Since 3 January 2020 when the first COVID19 cases was reported in Malaysia, by 1 Aug 2021, Malaysia has recorded 1,130,422 confirmed cases of COVID-19 with 9,184 deaths. Beginning on 18 March 2020, Government of Malaysia officially enforced the Movement Control Order (MCO) under the Prevention and Control of Infectious Diseases Act 1988 and the Police Act 1967. The order prohibited inter-state travel, gatherings and restrictions on the entry of all tourists and foreign visitors into the country. Although movement restrictions were relaxed when cases subsided, Government of Malaysia has reinstated the MCO 3.0 nationwide starting from 12 May 2021 have been extended beyond 28 June 2021 as cases continue to climb.  In mid-June 2021, the Government has launched the National Recovery Plan to guide, among other, the conditions for lifting the movement restrictions in the country.

Although the prolonged movement restrictions have extensively hindered the progress of the project’s implementation, the delays had been addressed by having activities moved onto the virtual space, with regular meetings and discussions with executing agencies in ensuring activities are implemented accordingly.

3.TE Purpose

The TE report will assess the achievement of project results against what was expected to be achieved, and draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming. The TE report promotes accountability and transparency, and assesses the extent of project accomplishments.

The TE will assess the project performance (in terms of relevance, effectiveness and efficiency), and determine outcomes and impacts (actual and potential) stemming from the project, including their sustainability between August – December 2021. The TE will have two primary purposes: (i) to provide evidence of results to meet accountability requirements, and (ii) to promote learning, feedback, and knowledge sharing through results and lessons learned among UNDP, Ministry of Water and Environment (KASA)[1], IP and their national project partners namely State DIDs (Perak, Sabah, Selangor and Wilayah Persekutuan), Global Environment Centre and Forever Sabah as well as project beneficiaries namely the local communities. The project is on its last year of the implementation cycle and the TE’s outcomes and recommendations will be instrumental to assess the impact of the project towards the development outcomes outlined in the Country Programme Action Plan 2016-2020. Lessons learned, best management practices and recommendations from the project will be used to inform the formulation, design and management of new UNDP pipeline projects in the Country Programme Action Plan 2022 – 2025, signed between Government of Malaysia and UNDP.

[1] Following the change in Government in February 2020, restructuring of the Ministry has resulted in the Department of Irrigation and Drainage to be placed under the Ministry of Environment and Water (KASA).

Duties and Responsibilities

DUTIES AND RESPONSIBILITIES

4. .TE Approach & Methodology

NOTE: In this section, incorporate any adjusted evaluative approaches/methodologies, as relevant, that may be needed to implement the evaluation effectively, including safety guidance, extended desk reviews, primary use of national consultants and virtual stakeholder meetings and virtual interviews by evaluators.   These methodologies and approaches plus any limitations faced during the TE process must be detailed in the TE Inception Report and final TE report

The TE must provide evidence-based information that is credible, reliable and useful.

The TE team will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Social and Environmental Screening Procedure/SESP) the Project Document, project reports including annual PIRs, project budget revisions, Independent Rapid Review report,  lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based evaluation. The TE team will review the baseline and terminal GEF focal area Core Indicators/Tracking Tools submitted to the GEF at the CEO endorsement and the terminal Core Indicators/Tracking Tools that must be completed before the TE field mission begins. 

The TE team is expected to follow a participatory and consultative approach ensuring close engagement with the Project Team, government counterparts (the GEF Political and Operational Focal Points), Implementing Partner, project partners, the UNDP Country Office, the Regional Technical Advisor, direct beneficiaries and other stakeholders.

Engagement of stakeholders is vital to a successful TE. Stakeholder involvement should include interviews with stakeholders who have project responsibilities, including but not limited to Department of Irrigation and Drainage (DID) under Ministry of Environment and Water, State DIDs agencies namely Perak, Sabah, Selangor and Wilayah Persekutuan, project partners (Global Environment Centre and Forever Sabah), senior officials and task team/component leaders, key experts and consultants in the subject area, Project Board, project beneficiaries, academia, local government and CSOs, etc.

The specific design and methodology for the TE should emerge from consultations between the TE team and the above-mentioned parties regarding what is appropriate and feasible for meeting the TE purpose and objectives and answering the evaluation questions, given limitations of budget, time and data. The TE team must, however, use gender-responsive methodologies and tools and ensure that gender equality and women’s empowerment, disability, as well as other cross-cutting issues and SDGs are incorporated into the TE report.

As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. Travel to the country has been restricted since 18 March 2020 travel in the country is also currently restricted. Given in-country travel is off limit, the TE team should develop a methodology that takes this into account the conduct of the TE virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. Such virtual methodology should be detailed in the TE Inception Report and agreed with the Commissioning Unit. 

If all or part of the TE is to be carried out virtually, consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/computer may be an issue.

The Lead Evaluator will work remotely with support by the National Evaluator in the field, if permissible, to travel for the purpose of the evaluation mission. No stakeholders, consultants or UNDP staff should be put in harm’s way and safety is the key priority. These limitations must be reflected in the final TE report.  The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the TE team.

The final TE report should describe the full TE approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses about the methods and approach of the evaluation.

5. Detailed Scope of the TE

The TE will assess project performance against expectations set out in the project’s Logical Framework/Results Framework (see TOR Annex A). The TE will assess results according to the criteria outlined in the Guidance for TEs of UNDP-supported GEF-financed Projects (http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEF-financedProjects.pdf).

The Findings section of the TE report will cover the topics listed below.

A full outline of the TE report’s content is provided in ToR Annex C.

The asterisk “(*)” indicates criteria for which a rating is required.

Findings

1. Project Design/Formulation

• National priorities and country driven-ness
• Theory of Change
• Gender equality and women’s empowerment
• Social and Environmental Safeguards
• Analysis of Results Framework: project logic and strategy, indicators
• Assumptions and Risks

• Lessons from other relevant projects (e.g. same focal area) incorporated into project design
• Planned stakeholder participation
• Linkages between project and other interventions within the sector
• Management arrangements

2. Project Implementation

• Adaptive management (changes to the project design and project outputs during implementation)
• Actual stakeholder participation and partnership arrangements
• Project Finance and Co-finance
• Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*)
• Implementing Agency (UNDP) (*) and Executing Agency (*), overall project oversight/implementation and execution (*)
• Risk Management, including Social and Environmental Standards

3. Project Results

• Assess the achievement of outcomes against indicators by reporting on the level of progress for each objective and outcome indicator at the time of the TE and noting final achievements
• Relevance (*), Effectiveness (*), Efficiency (*) and overall project outcome (*)
• Sustainability: financial (*), socio-political (*), institutional framework and governance (*), environmental (*), overall likelihood of sustainability (*)
• Country ownership
• Gender equality and women’s empowerment
• Cross-cutting issues (poverty alleviation, improved governance, climate change mitigation and adaptation, disaster prevention and recovery, human rights, capacity development, South-South cooperation, knowledge management, volunteerism, etc., as relevant)
• Identify complementarity between the results of this project and other projects in the Sustainable and Resilient Development Portfolio ie the “Improving Connectivity in the Central Forest Spine Landscape” Project (ATLAS ID 00077143/ PIMS 4594) and the new Malaysia’s FOLUR national project – PPG Heart of Borneo (ATLAS ID 00116884/ UNDP PIMS 6382).  
• GEF Additionality
• Catalytic Role / Replication Effect
• Progress to impact

4. Main Findings, Conclusions, Recommendations and Lessons Learned

• The TE team will include a summary of the main findings of the TE report. Findings should be presented as statements of fact that are based on analysis of the data.
•  The section on conclusions will be written in light of the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the TE findings. They should highlight the strengths, weaknesses and results of the project, respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries, UNDP and the GEF, including issues in relation to gender equality and women’s empowerment.
• Recommendations should provide concrete, practical, feasible and targeted recommendations directed to the intended users of the evaluation about what actions to take and decisions to make. The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation.
• The TE report should also include lessons that can be taken from the evaluation, including best and worst practices in addressing issues relating to relevance, performance and success that can provide knowledge gained from the particular circumstance (programmatic and evaluation methods used, partnerships, financial leveraging, etc.) that are applicable to other GEF and UNDP interventions. When possible, the TE team should include examples of good practices in project design and implementation.
• It is important for the conclusions, recommendations and lessons learned of the TE report to include results related to gender equality and empowerment of women.

The TE report outline, will including the Evaluation Ratings matrix, is enclosed in the as shown in the ToR Annex C and D.

6. Expected Outputs and Deliverables

The TE National Evaluator shall prepare and submit:

The final TE report must be in English. All final TE reports will be quality assessed by the UNDP Independent Evaluation Office (IEO).  Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines.[1]

7. TE Arrangements

NOTE: Detail the role of the Commissioning Unit and Project Team in supporting the implementation of remote/ virtual meetings. An updated stakeholder list with contact details (phone and email) will need to be provided by the Commissioning Unit to the TE team. Adjust the text if a mission will not take place.

The principal responsibility for managing the TE resides with the Commissioning Unit – UNDP Country Office.

The Project Team will be responsible for liaising with the TE team to provide all relevant documents, set up stakeholder interviews, and arrange field visits.

8. Duration of the Work

The total duration of the TE will be approximately 50 working days over a time period of 6 months starting 1 September 2021. The tentative TE timeframe is as follows:

The expected date start date of contract is 1 September 2021.

9. Duty Station

Identify the consultant’s duty station/location for the contract duration, mentioning ALL possible locations of field works/duty travel in pursuit of other relevant activities, specially where traveling to locations at security Phase I or above will be required.

Travel:

• Domestic travel will be required to the project sites in the States of Perak, Sabah, Selangor and Wilayah Persekutuan during the TE mission; the COVID-19 travel restrictions permitting.  Depending on the COVID-19 travel restrictions the missions may be organized physically or virtually;  
• The BSAFE course must be successfully completed prior to commencement of travel;
• Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director.
• Consultants are required to comply with the UN security directives set forth under: https://dss.un.org/dssweb/

10. Evaluator Ethics

The TE team will be held to the highest ethical standards and is required to sign a code of conduct upon acceptance of the assignment. This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The evaluator must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The evaluator must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.

11. Payment Schedule

• 20% payment upon satisfactory delivery of the final TE Inception Report and approval by the Commissioning Unit
• 40% payment upon satisfactory delivery of the draft TE report, and powerpoint slide deck to the Commissioning Unit
• 40% payment upon satisfactory delivery of the final TE report, presentation at the concluding workshop and approval by the Commissioning Unit and RTA (via signatures on the TE Report Clearance Form) and delivery of completed TE Audit Trail

Criteria for issuing the final payment of 40%

• The final TE report includes all requirements outlined in the TE TOR and is in accordance with the TE guidance.
• The final TE report is clearly written, logically organized, and is specific for this project (i.e. text has not been cut & pasted from other MTR reports).
• The Audit Trail includes responses to and justification for each comment listed.

In line with the UNDP’s financial regulations, when determined by the Commissioning Unit and/or the consultant that a deliverable or service cannot be satisfactorily completed due to the impact of COVID-19 and limitations to the TE, that deliverable or service will not be paid.

Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her control.

[1] Access at: http://web.undp.org/evaluation/guideline/section-6.shtml

Competencies

The selection of evaluators will be aimed at maximizing the overall “team” qualities in the following areas:

Corporate competencies:

• Commitment to UNDP’s mission, vision, values and ethical standards
• Sensitivity to cultural, gender, religion, race, nationality and age differences
• Treat all stakeholders fairly and without prejudice
• Maintains objectivity and impartiality in handling evaluation processes

Functional competencies:

• Experience in project development, implementation and evaluation particularly in directly managing results-based monitoring and evaluation methodologies
• Demonstrated experience in conducting evaluation of multi-year and multi-component programmes and projects
• Familiarity with the norms and issues in environmental conservation and management
• Demonstrated strong coordination and facilitation skills
• Strong interpersonal skills and the ability to initiate discussions with national/local governmental officials, civil society organizations and communities
• Demonstrated ability to function in a team environment and to deal with complex multi-stakeholder environment
• Demonstrated ability to prepare and present comprehensive reports

Required Skills and Experience

TE Team Composition and Required Qualifications

A team of two independent evaluators will conduct the TE – one international Lead Evaluator (with experience and exposure to projects and evaluations in other regions) and one National Evaluator expert from Malaysia or resides in Malaysia. The Lead Evaluator will be responsible for the overall design and writing of the TE report, etc. The National Evaluator will assess emerging trends with respect to regulatory frameworks, budget allocations, capacity building, carry out consultations in bahasa Malaysia as needed, provide support in translating key sections of documents from bahasa Malaysia to English as needed, provide written inputs to the evaluation report as required from the Team leader, and work with the Project Team in developing the TE itinerary, etc.

The evaluator(s) cannot have participated in the project preparation, formulation and/or implementation (including the writing of the project document), must not have conducted this project’s Mid-Term Review or Independent Rapid Review and should not have a conflict of interest with the project’s related activities.

The selection of evaluators will be aimed at maximizing the overall “team” qualities in the following areas:

Education

• A Master’s degree or higher in biodiversity, conservation biology, ecology, limnology, environmental or natural resources management, water resource management or other closely related field (10 points)

Experience

• Relevant experience with results-based management evaluation methodologies for at least 10 years (15 points);
• Working experience in river ecology, water resource management and environmental related areas for at least 10 years (10 points);
• Experience applying SMART indicators and/or reconstructing/validating baseline scenarios (10 points);
• Experience working with UNDP or GEF or within UN systems evaluations for at least 5 years (10 points);
• Experience working in South-East Asian, Asia-Pacific or other region in the area of environment (5 points)
• Demonstrated understanding of issues related to gender and disability, with experience in gender/disability responsive evaluation and analysis (10 points);

Language

• Fluency in written and spoken English and Bahasa Malaysia.

DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE PROPOSALS

Interested individual applicants must submit the following documents/information to demonstrate their qualifications:

1. Document 1 : Letter of Confirmation of Interest and Availability at https://undp.org/content/dam/malaysia/docs/Procurement/Template%20of%20Letter%20of%20Interest.docx
2. Document 2: Technical Proposal: of why the individual considers him/herself as the most suitable for the assignment, and a proposed methodology on how they will approach and complete the assignment; (max 1 page)
3. Document 3: Financial Proposal at https://undp.org/content/dam/malaysia/docs/Procurement/Financial%20Proposal%20Template-Deliverables_MYR_IC2021-031.doc that indicates the all-inclusive fixed total contract price and all other travel related costs (such as flight ticket, per diem, etc.), supported by a breakdown of costs, as per template attached to the Letter of Confirmation of Interest template. If an applicant is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the applicant must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.
4. Document 4: Personal CV or P11 indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references  https://undp.org/content/dam/malaysia/docs/Procurement/P11%20for%20Consultant_Optional.docx
5. Annexes to the Terminal Evaluation:-
   • Annex A:   https://undp.org/content/dam/malaysia/docs/Procurement/Annex%20A%20PIMS%205281_Revised%20Strategic%20Results%20Framework.pdf
   • Annex B to H:  https://undp.org/content/dam/malaysia/docs/Procurement/Annex%20B%20to%20H.docx
6. Please submit this information in one file as Document 1, 2, 3 and 4.The system will only accept one attachment , please merge all documents and submit as one file.
7. Applicants must reply to the mandatory questions asked by the system when submitting the application.
8. Candidates who fail to submit all the information requested above will be disqualified.

How to Apply:

• Kindly download the Letter of Confirmation of interest and availability, Financial Proposal Template and General Terms & Conditions mentioned below;
• Read and agree to the General Terms & Conditions (refer to below link);
• Click the ‘apply’ icon and complete what is required;
• Scan all documents into 1 pdf folder and then upload;
• For clarification question, please email to procurement.my@undp.org. The clarification question deadline is three (3) days before the closing. When emailing for clarification questions, please put "MyIC/2021/031"as the subject matter.

General terms & conditions to be downloaded:

• The General Terms & Conditions for Individual contract is available at: https://www.my.undp.org/content/dam/malaysia/docs/Procurement/AC_AntiFraud_General%20Conditions%20of%20Contract%20for%20Individual%20Contracts%20(2).pdf

Reimbursable Loan Agreement (RLA): A legal instrument between UNDP and a Company/institution, according to which, the latter makes available the services of an individual delivering time-bound and quantifiable outputs that are directly linked to payments

• The General Terms & Conditions for Reimbursement Loan Agreement is available at:https://www.my.undp.org/content/dam/malaysia/docs/Procurement/Reimbursable%20Loan%20Agreement%20_%20Terms%20&%20Conditions.pdf

Incomplete proposals may not be considered. Only short-listed candidates may be contacted, and successful candidates will be notified.

Criteria for Selection of the Best Offer

Only those applications which are responsive and compliant will be evaluated. Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the price proposal will weigh as 30% of the total scoring. The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.