Background

INTRODUCTION

In accordance with UNDP and GEF M&E policies and procedures, all full and medium-sized UNDP support GEF financed projects are required to undergo a terminal evaluation upon completion of implementation. These terms of reference (TOR) sets out the expectations for a Terminal Evaluation (TE) of Promoting Renewable Energy in Mae Hong Son Province (PIMS #3908)’.

PROJECT SUMMARY TABLE

Duties and Responsibilities

PURPOSE, OBJECTIVE AND SCOPE:

The Project Objective is “to overcome barriers to the provision of Renewable Energy (RE) services in integrated provincial renewable energy programmes in Thailand”.  This will contribute to the broader Goal of reducing GHG emissions in Thailand. Importantly, it will also contribute to the Goal of Thailand’s GEF strategy, which is to mobilize GEF resources in support of the implementation of Sufficiency Economy principles, as enshrined in the 10th National Economic and Social Development Plan.

Following a Mid-Term-Review (MTR) in Q3 of 2013, significant changes were made to the project framework and the implementation modality. In the second phase of the project the focus is more on off-grid renewable energy applications and the project implementation modality is Direct Implementation (DIM).                    

The second phase of the project aims at facilitating an integrated  RE  planning process at provincial and local level, in order to translate targets set at national level to local level and into real action. The four components of the project focus on (a) institutional capacity development for planning and implementing RE programmes; (b) access to financing; (c) technical training and education and (d) policies for up-scaling and replication.

 In order to realize the project objective, the project was designed to comprise of four components, each of which addressing a specific category of barriers to renewable energy development in MHS. The project components and outputs for the remaining period of the project are:

The RE technology focussed during the second phase of the project has 7 items:

1. Off-grid micro-hydropower
2. On-grid solar farm
3. Solar home system (SHS) rehabilitation and solar lanterns
4. Improved cookstoves (ICS)
5. Provincial integrated RE planning
6. Solar rooftop and Energy Efficiency measures in government buildings
7. Biodigesters

Described in the Addendum of the Project Document that at the end of the second phase of the project, the following are the expected outcomes on the ground:

• 1 on-grid solar farm project approved, installed and operational (500 kW);
• 100 SHS rehabilitated (100*120 Wp);
• 200 solar lanterns sold (200*2.5W);
• 20 additional biodigesters at schools, SMEs and farms installed and operational;
• 2 off-grid hydropower plants approved, installed and operational (2*30 kW);
• 10 solar rooftop systems approved, installed and operational (10*200 W);
• 1 EE project in gov. building approved, implemented and operational (RE capacity 600 W savings);
• 10 villages in which ICS have been tried out and being used in MHS by end of 2016 (50 systems).
• Direct reduction of GHG emissions due to operation of these systems is about 14,216 tonnes CO2.

IN 2016, due to development complexities on the ground, several project results were modified. These included the unattained micro-hydro power (MHP), solar farm and solar rooftop. Below are the new agreed results for the 2017 project extension period endorsed by the Project Board on 25 May 2016, and later by UNDP CO and the regional office on 14 November 2016:

The total project budget is USD. US$ 2,712,700.

The TE will be conducted according to the guidance, rules and procedures established by UNDP and GEF as reflected in the UNDP Evaluation Guidance for GEF Financed Projects.

The objectives of the evaluation are to assess the achievement of projects results, and to draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming.

EVALUATION APPROACH AND METHOD

An overall approach and method for conducting project terminal evaluations of UNDP supported GEF financed projects has developed over time. The evaluator is expected to frame the evaluation effort using the criteria of relevance, effectiveness, efficiency, sustainability, and impact, as defined and explained in the UNDP Guidance for Conducting Terminal Evaluations of UNDP-supported, GEF-financed Projects.    A set of questions covering each of these criteria have been drafted and are included with this TOR (Annex C). The evaluator is expected to amend, complete and submit this matrix as part of an evaluation inception report, and shall include it as an annex to the final report.

The evaluation must provide evidence-based information that is credible, reliable and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, in particular the GEF operational focal point, UNDP Country Office, project team, UNDP GEF Technical Adviser based in the region and key stakeholders. The evaluator is expected to conduct a field mission to Thailand including the project sites in Mae Hong Son (MHS) province.

At the project sites, key stakeholders include MHS Provincial Office, MHS Provincial Energy Office, the local governments, schools and communities should be interviewed.

Interviews will be held with the following personnel and organizations and individuals at a minimum:

Representative of Responsible Parties, including:

• Governor of MHS
• Chief of MHS Provincial Office
• Chief of MHS Provincial Energy Office
• Members of the Project Board
• Chiefs of Tambon (sub-district) Administrative Organizations (local governments)
• Directors of school, hospitals
• Representatives from target communities
• Representative from the key service provider of the project

Project Team

• Project Manager
• Project Field Officer
• Project Assistant

UNDP Country Office in Bangkok in-charge of this project. UNDP:

• BRH Regional Technical Specialist,
• IGSD/ UNDP Thailand Programme Manager

The evaluator will review all relevant sources of information, such as the project document, project reports – including Annual APR/PIR, project budget revisions, midterm review, progress reports, GEF focal area tracking tools, project files, national strategic and legal documents, and any other materials that the evaluator considers useful for this evidence-based assessment. A list of documents that the project team will provide to the evaluator for review is included in Annex B of this Terms of Reference. The full scope methods used in the evaluation are at the discretion of the evaluator(s), but a mixed method of document review, interviews, and direct observations should be employed, at a minimum. The TE inception report and TE report should explain all the evaluation methods used in detail.

EVALUATION CRITERIA & RATINGS

An assessment of project performance will be carried out, based against expectations set out in the Project Logical Framework/Results Framework (see  Annex A), which provides performance and impact indicators for project implementation along with their corresponding means of verification. The evaluation will at a minimum cover the criteria of: relevance, effectiveness, efficiency, sustainability and impact. Ratings must be provided on the following performance criteria. The completed table must be included in the evaluation executive summary.   The obligatory rating scales are included in  Annex D.

PROJECT FINANCE / COFINANCE

The Evaluation will assess the key financial aspects of the project, including the extent of co-financing planned and realized.  Project cost and funding data will be required, including annual expenditures.  Variances between planned and actual expenditures will need to be assessed and explained.  Results from recent financial audits, as available, should be taken into consideration. The evaluator(s) will receive assistance from the Country Office (CO) and Project Team to obtain financial data in order to complete the co-financing table below, which will be included in the terminal evaluation report.

MAINSTREAMING

UNDP supported GEF financed projects are key components in UNDP country programming, as well as regional and global programmes. The evaluation will assess the extent to which the project was successfully mainstreamed with other UNDP priorities, including poverty alleviation, improved environment, governance, and gender.

IMPACT

The evaluators will assess the extent to which the project is achieving impacts or progressing towards the achievement of impacts. Key findings that should be brought out in the evaluations include whether the project has demonstrated: a) verifiable improvements in ecological status, b) verifiable reductions in stress on ecological systems, and/or c) demonstrated progress towards these impact achievements.

CONCLUSIONS, RECOMMENDATIONS & LESSONS

The evaluation report must include a chapter providing a set of conclusions, recommendations and lessons.

IMPLEMENTATION ARRANGEMENTS

The principal responsibility for managing this evaluation resides with the UNDP CO in Thailand.  The UNDP CO will contract the evaluators and ensure the timely provision of per diems and travel arrangements within the country for the evaluation team. The Project Team will be responsible for liaising with the Evaluators team to set up stakeholder interviews, arrange field visits, coordinate with the Government etc.

EVALUATION TIMEFRAME

The total duration of the evaluation will be 19 days over a time period from 1 September to 30 November 2017 according to the following plan:

EVALUATION DELIVERABLES

The evaluation team is expected to deliver the following:

*When submitting the final evaluation report, the evaluator is required also to provide an 'audit trail', detailing how all received comments have (and have not) been addressed in the final evaluation report. See Annex H for an audit trail template.

DUTY STATION

Home-based with travel to Bangkok and Mae Hong Son, Thailand

Competencies

Corporate Competencies:

• Demonstrates integrity by modelling the UN’s values and ethical standards;
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability;
• Treats all people fairly without favouritism.

Technical Competencies:

• Analytic capacity and demonstrated ability to process, analyse and synthesise complex, technical information;
• Proven ability to support the development of high quality knowledge and training materials, and to train technical teams;
• Proven experience in the developing country context and working in different cultural settings.

 Communication:

• Communicate effectively in writing to a varied and broad audience in a simple and concise manner.

 Professionalism:

• Capable of working in a high pressure environment with sharp and frequent deadlines, managing many tasks simultaneously;
• Excellent analytical and organizational skills.

 Teamwork:

• Projects a positive image and is ready to take on a wide range of tasks;
• Focuses on results for the client;
• Welcomes constructive feedback.

Required Skills and Experience

TEAM COMPOSITION

The evaluation team will be composed of an international and a national evaluator.  The consultants shall have prior experience in evaluating similar projects.  Experience with GEF financed projects is an advantage.  The international evaluator will be designated as the team leader and will be responsible for finalizing the report.  The evaluators selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

The team members must present the following qualifications:

INTERNATIONAL LEAD CONSULTANT 

Education:

• Post-Graduate in energy, environmental studies, engineering, development studies, social sciences and/ or other related fields (15%)
• Experience:
• Minimum of 8 years accumulated and recognized experience in the field of energy policy, rural energy development planning, sustainable development (20%)
• Minimum of 5 years of project evaluation and/or implementation experience in the result-based management framework, adaptive management and UNDP or GEF Monitoring and Evaluation Policy (20%)
• Familiarity in similar country or regional situations relevant to that of Promoting Renewable Energy in Mae Hong Son Project is an advantage (5%).
• Experience with multilateral and bilateral supported renewable energy, sustainable realization and utilisation of RE technologies (10%)
• Comprehensive knowledge of international best practices in renewable energy, poverty reduction and sustainable development (15%)
• Language requirement:
• Excellent written English (15%)

Responsibilities

• Documentation review
• Leading the TE Team in planning, conducting and reporting on the evaluation
• Deciding on division of labour within the Team and ensuring timeliness of reports
• Use of best practice evaluation methodologies in conducting the evaluation
• Leading the drafting and finalization of the Inception Report for the Terminal Evaluation
• Leading presentation of the draft evaluation findings and recommendations in-country
• Conducting the de-briefing for the UNDP Country Office in Thailand and Core Project Management Team
• Leading the drafting and finalization of the Terminal Evaluation Report

FINANCIAL PROPOSAL

Payment modalities and specifications

EVALUATION

EVALUATOR ETHICS 

Evaluation consultants will be held to the highest ethical standards and are required to sign a Code of Conduct (Annex E) upon acceptance of the assignment. UNDP evaluations are conducted in accordance with the principles outlined in the UNEG 'Ethical Guidelines for Evaluations'.

Criteria for Evaluation of Proposal:

Only those applications which are responsive and compliant will be evaluated.  Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the price proposal will weigh as 30% of the total scoring.  The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.

DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE PROPOSALS:

Interested individual consultants must submit the following documents/information to demonstrate their qualifications. Please group them into one(1) single PDF document as the application only allows to upload maximum one document:

1. Letter of Confirmation of Interest and Availability using the template provided by UNDP;
2. CV and a Personal History Form (P11 form);
3. Brief description of approach to work/technical proposal of why the individual considers him/herself as the most suitable for the assignment, and a proposed methodology on how they will approach and complete the assignment; (max 1 page)
4. Financial Proposal that indicates the all-inclusive fixed total contract price and all other travel related costs (such as flight ticket, per diem, etc.), supported by a breakdown of costs, as per template attached to the Letter of Confirmation of Interest template.  If an applicant is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the applicant must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.

 All application materials should be submitted by CoB 26 April 2017.  Incomplete applications will be excluded from further consideration.

Annexes:

• Annex I - TOR - MHS-RE_Terminal Evaluation (International Consultant): http://procurement-notices.undp.org/view_file.cfm?doc_id=105876
• Annex II -    General Condition of Contract : http://procurement-notices.undp.org/view_file.cfm?doc_id=105684
• Annex III – Offeror’s Letter to UNDP Confirming Interest and Availability for the Individual IC : http://procurement-notices.undp.org/view_file.cfm?doc_id=105877

All documents can be downloaded at http://procurement-notices.undp.org/view_notice.cfm?notice_id=36917