Antecedentes

1. INTRODUCTION

This is the Terms of Reference (ToR) for the UNDP-GEF Midterm Review (MTR) of the full -sized project titled Building Shoreline Resilience of Timor-Leste to Protect Local Communities and their Livelihoods (PIMS#5330) implemented by UNDP Timor-Leste in collaboration with Ministry of Agriculture and Fisheries (MAF), which is to be undertaken in four years (2016 - 2020). The project started on the 19th August 2016 and is in its third year of implementation. In line with the UNDP-GEF Guidance on MTRs, this MTR process was initiated before the submission of the second Project Implementation Report (PIR). This ToR sets out the expectations for this MTR.  The MTR process must follow the guidance outlined in the document Guidance for Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects http://web.undp.org/evaluation/guidance.shtml

2.  PROJECT BACKGROUND INFORMATION

The project was designed to: strengthen resilience of coastal communities by the introduction of nature-based approaches to coastal protection. The project is supporting inter- and intra-ministerial coordination for collaborative development planning ensuring protection of coastal areas, as well as identify and research potential revenue streams for long term sustainability.

The project is under implementation in close collaboration with the Government of Timor-Leste (GoTL) through the Ministry of Agriculture and Fisheries (MAF), as well as in partnership with Ministry of Commerce, Industry and Environment; Ministry of Public Works, Ministry of Social Solidarities, UNTL, NGOs and private enterprises.  As the issues of coastal areas are complex and cross-sectoral, the project employs an integrated approach, while tailoring activities to address the specific needs, challenges and priorities of the Government of Timor Leste.

As mangroves are vital natural defense to the impacts of climate change, extensive mangrove protection and restoration will be supported while addressing community pressures (i.e. felling for fuelwood) and introduce alternative mangrove-supportive livelihoods, as well as improve public awareness about the important role of coastal ecosystems in shoreline protection and climate change adaptation and mitigation.

Since Timor-Leste’s landscape is generally steep, therefore, where relevant, the project looks at upland SLM activities to reduce impacts of sedimentation, increased runoff and flash floods, and availability of groundwater of the coastal areas.

Mangroves and coastal wetlands are highly vulnerable to the projected impacts of climate change, particularly sea-level rise (SLR). An estimated 80% of mangroves have been lost in Timor Leste, since 1940 (Boggs et al. 2009, Alongi 2014), due to a combination of both, climate related risks (i.e. sea level rise, increased storm events) and also, non-climate related anthropogenic impacts (i.e. demand for fuel wood, building materials, salt production, rice production, uncontrolled grazing).  While the relative contribution of these causal, climate and non-climate factors is unknown, anecdotal evidence (i.e. the lack of natural regeneration in many areas), suggest SLR has had major impacts, particularly on the north coast.  The loss of mangrove forests has resulted in greater exposure to sea surges, inundation, erosion and accretion processes.

These pressures from upland areas, coupled with the rapidly rising sea level, are putting coastal communities (and the ecosystems and resources upon which they depend), particularly at risk. Over the past two decades, mangroves, which serve as a natural defense to the sea, have been severely degraded – leaving the country’s shoreline and coastal communities vulnerable to coastal inundation, erosion, salt water intrusion, and impacts of sea-borne natural hazards (e.g. waves, storm surges, and in extreme cases, small scale tsunamis).

The major non-climate impacts on mangroves include large-scale, land clearance and conversion for rice farming and traditional salt production, and also, their use as fuelwood, for cooking and household income. Mangroves are also illegally cut for house construction, boatbuilding, and also, for fuel wood to support traditional salt-making livelihood activities.

With high levels of food insecurity, limited cash income and limited knowledge of climate risks, the coastal protection benefits of mangroves, and broader ecosystem goods and services (EGS) benefits of mangroves, there are currently very limited incentives for coastal communities in Timor Leste to protect and conserve mangroves.  Further, restoration projects to-date have been short-term – too short for community learnings to take place, and for mangroves to have time to regenerate, before the project stopped paying and the community stopped protecting.

Communities are currently not guided or provided with sufficient incentives to become stewards of natural resources, ecosystems or the essential services that grant coastal protection and livelihood sustainability. There are sporadic interventions by the government to improve the situation in mangrove areas, like in Ulmera village where mangrove rehabilitation and replanting have been piloted to cover 3km2 mangrove area. But rarely are mangrove rehabilitation and livelihood development linked to achieve sustainable results. Employment and income generation potential associated with mangrove rehabilitation, protection and sustainable management has not been exploited as part of the local, suco (smallest administrative area) level development plans, investments or public and private partnership initiatives.

There is limited knowledge about the win-win solutions, whereby protection of natural assets such as mangroves can effectively protect and sustain physical and economic assets against climate change induced hazards and at the same time deliver on social and economic benefits.

This project will systematically strengthen the synergistic relationship between coastal communities and mangroves ecosystems and ensure that coastal communities in Timor Leste have economic incentives to maintain and safeguard these protective natural systems, without compromising their livelihood options.  This will be achieved through community-led adaptation interventions, that include mangrove re-afforestation, conservation and livelihood diversification options (such as agroforestry, fish ponds, intensive gardening, fruit trees, developed through integrated community-based land use models and adaptation plans (such as the Forest-Fish-Fruit mound-ditch model, successfully implemented in Bangladesh.

3.  OBJECTIVES OF THE MTR

The MTR will assess progress towards the achievement of the project objectives and outcomes as specified in the Project Document and assess early signs of project success or failure with the goal of identifying the necessary changes to be made in order to set the project on-track to achieve its intended results. The MTR will also review the project’s strategy, its risks to sustainability.

4. MTR APPROACH & METHODOLOGY 

The MTR must provide evidence-based information that is credible, reliable and useful. The MTR team will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Environmental & Social Safeguard Policy, the Project Document, project reports including Annual Project Review/PIRs, project budget revisions, lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based review). The MTR team will review the baseline GEF focal area Tracking Tool submitted to the GEF at CEO endorsement, and the midterm GEF focal area Tracking Tool that must be completed before the MTR field mission begins.  

The MTR team is expected to follow a collaborative and participatory approach[1] ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), the UNDP Country Office(s), UNDP-GEF Regional Technical Advisers, and other key stakeholders.

Engagement of stakeholders is vital to a successful MTR.[2] Stakeholder involvement should include interviews with stakeholders who have project responsibilities, including but not limited to different Directorate of Ministry of Agriculture and Fisheries(MAF); Ministry of Commerce, Industry and Environment(MCIE), Ministry of Public Works(MPW), University of Timor-Leste(UNTL), executing agencies/ NGOs, senior officials and task team/ component leaders, key experts and consultants in the subject area, Project Board, project stakeholders, academia, local government and CSOs, etc. Additionally, the MTR consultant is expected to conduct field missions to Dili, Liquica, Manatuto, Viqueque, Manufahi, Covalima and Bobonaro municipalities, including the following project sites Uatukurbao, Uaniuma, Aubeon, Modomahut, Fatukahi, Mahakidan, Dotic, Betano, Selele-Boot, Suai-Loro, Be-malai, Beacou, Lake-Mobara, Ulmera, Hera, Metinaro among others.

The final MTR report should describe the full MTR approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses about the methods and approach of the review.

[1] For ideas on innovative and participatory Monitoring and Evaluation strategies and techniques, see UNDP Discussion Paper: Innovations in Monitoring & Evaluating Results, 05 Nov 2013.

[2] For more stakeholder engagement in the M&E process, see the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results, Chapter 3, pg. 93.

Deberes y responsabilidades

5.  DETAILED SCOPE OF THE MTR

The MTR team will assess the following four categories of project progress. See the Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for extended descriptions.

i.    Project Strategy

Project design:

• Review the problem addressed by the project and the underlying assumptions.  Review the effect of any incorrect assumptions or changes to the context to achieving the project results as outlined in the Project Document.
• Review the relevance of the project strategy and assess whether it provides the most effective route towards expected/intended results.  Were lessons from other relevant projects properly incorporated into the project design?
• Review how the project addresses country priorities. Review country ownership. Was the project concept in line with the national sector development priorities and plans of the country (or of participating countries in the case of multi-country projects)?
• Review decision-making processes: were perspectives of those who would be affected by project decisions, those who could affect the outcomes, and those who could contribute information or other resources to the process, taken into account during project design processes?
• Review the extent to which relevant gender issues were raised in the project design. See Annex 9 of Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for further guidelines.
• If there are major areas of concern, recommend areas for improvement.

Results Framework/Log frame:

• Undertake a critical analysis of the project’s log frame indicators and targets, assess how “SMART” the midterm and end-of-project targets are (Specific, Measurable, Attainable, Relevant, Time-bound), and suggest specific amendments/revisions to the targets and indicators as necessary.
• Are the project’s objectives and outcomes or components clear, practical, and feasible within its time frame?
• Examine if progress so far has led to or could in the future catalyse beneficial development effects (i.e. income generation, gender equality and women’s empowerment, improved governance etc...) that should be included in the project results framework and monitored on an annual basis.
• Ensure broader development and gender aspects of the project are being monitored effectively.  Develop and recommend SMART ‘development’ indicators, including sex-disaggregated indicators and indicators that capture development benefits.

ii.    Progress Towards Results

Progress Towards Outcomes Analysis:

• Review the log frame indicators against progress made towards the end-of-project targets using the Progress Towards Results Matrix and following the Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects; colour code progress in a “traffic light system” based on the level of progress achieved; assign a rating on progress for each outcome; make recommendations from the areas marked as “Not on target to be achieved” (red).

Table. Progress Towards Results Matrix (Achievement of outcomes against End-of-project Targets)

Indicator Assessment Key

In addition to the progress towards outcomes analysis:

• Compare and analyse the GEF Tracking Tool at the Baseline with the one completed right before the Midterm Review.
• Identify remaining barriers to achieving the project objective in the remainder of the project.
• By reviewing the aspects of the project that have already been successful, identify ways in which the project can further expand these benefits.

iii.   Project Implementation and Adaptive Management

Management Arrangements:

• Review overall effectiveness of project management as outlined in the Project Document.  Have changes been made and are they effective?  Are responsibilities and reporting lines clear?  Is decision-making transparent and undertaken in a timely manner?  Recommend areas for improvement.
• Review the quality of execution of the Executing Agency/Implementing Partner(s) and recommend areas for improvement.
• Review the quality of support provided by the GEF Partner Agency (UNDP) and recommend areas for improvement.

Work Planning:

• Review any delays in project start-up and implementation, identify the causes and examine if they have been resolved.
• Are work-planning processes results-based?  If not, suggest ways to re-orientate work planning to focus on results?
• Examine the use of the project’s results framework/ logframe as a management tool and review any changes made to it since project start. 

Finance and co-finance:

• Consider the financial management of the project, with specific reference to the cost-effectiveness of interventions. 
• Review the changes to fund allocations as a result of budget revisions and assess the appropriateness and relevance of such revisions.
• Does the project have the appropriate financial controls, including reporting and planning, that allow management to make informed decisions regarding the budget and allow for timely flow of funds?
• Informed by the co-financing monitoring table to be filled out, provide commentary on co-financing: is co-financing being used strategically to help the objectives of the project? Is the Project Team meeting with all co-financing partners regularly in order to align financing priorities and annual work plans?

Project-level Monitoring and Evaluation Systems:

• Review the monitoring tools currently being used:  Do they provide the necessary information? Do they involve key partners? Are they aligned or mainstreamed with national systems?  Do they use existing information? Are they efficient? Are they cost-effective? Are additional tools required? How could they be made more participatory and inclusive?
• Examine the financial management of the project monitoring and evaluation budget.  Are sufficient resources being allocated to monitoring and evaluation? Are these resources being allocated effectively?

Stakeholder Engagement:

• Project management: Has the project developed and leveraged the necessary and appropriate partnerships with direct and tangential stakeholders?
• Participation and country-driven processes: Do local and national government stakeholders support the objectives of the project?  Do they continue to have an active role in project decision-making that supports efficient and effective project implementation?
• Participation and public awareness: To what extent has stakeholder involvement and public awareness contributed to the progress towards achievement of project objectives?

Reporting:

• Assess how adaptive management changes have been reported by the project management and shared with the Project Board.
• Assess how well the Project Team and partners undertake and fulfil GEF reporting requirements (i.e. how have they addressed poorly-rated PIRs, if applicable?)
• Assess how lessons derived from the adaptive management process have been documented, shared with key partners and internalized by partners.

Communications:

• Review internal project communication with stakeholders: Is communication regular and effective? Are there key stakeholders left out of communication? Are there feedback mechanisms when communication is received? Does this communication with stakeholders contribute to their awareness of project outcomes and activities and investment in the sustainability of project results?
• Review external project communication: Are proper means of communication established or being established to express the project progress and intended impact to the public (is there a web presence, for example? Or did the project implement appropriate outreach and public awareness campaigns?)
• For reporting purposes, write one half-page paragraph that summarizes the project’s progress towards results in terms of contribution to sustainable development benefits, as well as global environmental benefits.

iv.   Sustainability

• Validate whether the risks identified in the Project Document, Annual Project Review/PIRs and the ATLAS Risk Management Module are the most important and whether the risk ratings applied are appropriate and up to date. If not, explain why.
• In addition, assess the following risks to sustainability:

Financial risks to sustainability:

• What is the likelihood of financial and economic resources not being available once the GEF assistance ends (consider potential resources can be from multiple sources, such as the public and private sectors, income generating activities, and other funding that will be adequate financial resources for sustaining project’s outcomes)?

Socio-economic risks to sustainability:

• Are there any social or political risks that may jeopardize sustainability of project outcomes? What is the risk that the level of stakeholder ownership (including ownership by governments and other key stakeholders) will be insufficient to allow for the project outcomes/benefits to be sustained? Do the various key stakeholders see that it is in their interest that the project benefits continue to flow? Is there sufficient public / stakeholder awareness in support of the long-term objectives of the project? Are lessons learned being documented by the Project Team on a continual basis and shared/ transferred to appropriate parties who could learn from the project and potentially replicate and/or scale it in the future?

Institutional Framework and Governance risks to sustainability:

• Do the legal frameworks, policies, governance structures and processes pose risks that may jeopardize sustenance of project benefits? While assessing this parameter, also consider if the required systems/ mechanisms for accountability, transparency, and technical knowledge transfer are in place.

Environmental risks to sustainability:

• Are there any environmental risks that may jeopardize sustenance of project outcomes?

Conclusions & Recommendations

The MTR team will include a section of the report setting out the MTR’s evidence-based conclusions, in light of the findings.[6]

Recommendations should be succinct suggestions for critical intervention that are specific, measurable, achievable, and relevant. A recommendation table should be put in the report’s executive summary. See the Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for guidance on a recommendation table.

The MTR team should make no more than 15 recommendations total.

Ratings

The MTR team will include its ratings of the project’s results and brief descriptions of the associated achievements in a MTR Ratings & Achievement Summary Table in the Executive Summary of the MTR report. See Annex E for ratings scales. No rating on Project Strategy and no overall project rating is required.

Table. MTR Ratings & Achievement Summary Table for Building Shoreline Resilience of Timor-Leste to Protect Local Communities and their Livelihoods Project

1. TIMEFRAME

The total duration of the MTR will be approximately 30 working days starting 01 October 2018; 14 days in Timor-Leste and 16 days home-based.

The tentative MTR timeframe is as follows:

Options for site visits should be provided in the Inception Report.

1. MIDTERM REVIEW DELIVERABLES

*The final MTR report must be in English. If applicable, the Commissioning Unit may choose to arrange for a translation of the report into a language more widely shared by national stakeholders.

1. MTR ARRANGEMENTS

The principal responsibility for managing this MTR resides with the Sustainable Development Unit, UNDP Timor-Leste country office.

The Sustainable Unit UNDP Timor-Leste country office will contract the consultants and ensure the timely provision of per diems and/or travel arrangements in Timor-Leste for the MTR consultant. The Project Team will be responsible for liaising with the MTR consultant to provide all relevant documents, set up stakeholder interviews, and arrange field visits.

[1] Populate with data from the Logframe and scorecards

[2] Populate with data from the Project Document

[3] If available

[4] Colour code this column only

[5] Use the 6-point Progress Towards Results Rating Scale: HS, S, MS, MU, U, HU

[6] Alternatively, MTR conclusions may be integrated into the body of the report.

Competencias

1.  TEAM COMPOSITION

A team of two independent consultants (one international and one national) will conduct the MTR. The International Consultant will be the team leader (with experience and exposure to projects and evaluations in other regions globally) and one team expert, usually from the country of the project (national consultant).  The consultants cannot have participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) and should not have a conflict of interest with project’s related activities. 

The selection of consultants will be aimed at maximizing the overall “team” qualities in the following areas and both international and national consultants must have:  

• Recent experience with result-based management evaluation methodologies;
• Experience applying SMART indicators and reconstructing or validating baseline scenarios;
• Competence in adaptive management, as applied to GEF, climate change, biodiversity and other relevant Focal Area);
• Experience working with the GEF or GEF-evaluations;
• Experience working in small island states preferably in Asia and Pacific region
• Work experience in relevant technical areas for at least 10 years;
• Demonstrated understanding of issues related to gender and climate change, biodiversity and other relevant Focal Area; experience in gender sensitive evaluation and analysis.
• Excellent communication skills;
• Demonstrable analytical skills;
• Project evaluation/review experiences within United Nations system will be considered an asset;
• A Master’s degree in Ecology, Coastal Ecosystem Management, Natural Resources Management, Environmental Science, Climate Change Adaptation, or in any disciplines relevant to Mangrove/coastal ecosystem restoration, coastal adaptation and shoreline management.), or other closely related field. Only candidates obtaining a minimum of 49 points in the technical evaluation criteria would be considered for the Financial Evaluation

1. PAYMENT MODALITIES AND SPECIFICATIONS

Habilidades y experiencia requeridas

1. APPLICATION PROCESS[1]

Recommended Presentation of Proposal: 

1. Letter of Confirmation of Interest and Availability using the template[2] provided by UNDP;
2. CV and a Personal History Form (P11 form[3]);
3. Brief description of approach to work/technical proposal of why the individual considers him/herself as the most suitable for the assignment, and a proposed methodology on how they will approach and complete the assignment; (max 1 page)
4. Financial Proposal that indicates the all-inclusive fixed total contract price and all other travel related costs (such as flight ticket, per diem, etc), supported by a breakdown of costs, as per template attached to the Letter of Confirmation of Interest template.  If an applicant is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the applicant must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP. 

All application materials should be submitted to the address (fill address) in a sealed envelope indicating the following reference “Consultant for (project title) Midterm Review” or by email at the following address ONLY: (fill email) This email address is being protected from spam bots, you need Javascript enabled to view it by (time and date). Incomplete applications will be excluded from further consideration.

Criteria for Evaluation of Proposal:  Only those applications which are responsive and compliant will be evaluated.  Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the price proposal will weigh as 30% of the total scoring.  The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.

ToR ANNEX A: List of Documents to be reviewed by the MTR Team

1. PIF
2. UNDP Initiation Plan
3. UNDP Project Document
4. UNDP Environmental and Social Screening results
5. Project Inception Report
6. All Project Implementation Reports (PIR’s)
7. Quarterly progress reports and work plans of the various implementation task teams
8. All monitoring reports prepared by the project
9. Financial and Administration guidelines used by Project Team

The following documents will also be available:

1. Project operational guidelines, manuals and systems
2. UNDP country/countries programme document(s)
3. Minutes of the Building Shoreline Resilience of Timor-Leste to Protect Local Communities and their Livelihoods Board Meetings and other meetings (i.e. Project Appraisal Committee meetings)
4. Project site location maps

ToR ANNEX B: Guidelines on Contents for the Midterm Review Report[4]

ToR ANNEX C: Midterm Review Evaluative Matrix Template

ToR ANNEX D: UNEG Code of Conduct for Evaluators/Midterm Review Consultants[5]

ToR ANNEX E: MTR Ratings

ToR ANNEX F: MTR Report Clearance Form

(to be completed by the Commissioning Unit and UNDP-GEF RTA and included in the final document)

[1] Engagement of the consultants should be done in line with guidelines for hiring consultants in the POPP: https://info.undp.org/global/popp/Pages/default.aspx

[2] https://intranet.undp.org/unit/bom/pso/Support%20documents%20on%20IC%20Guidelines/Template%20for%20Confirmation%20of%20Interest%20and%20Submission%20of%20Financial%20Proposal.docx

[3] http://www.undp.org/content/dam/undp/library/corporate/Careers/P11_Personal_history_form.doc

[4] The Report length should not exceed 40 pages in total (not including annexes).

[5] www.undp.org/unegcodeofconduct