Background

The Project Document was signed between the Lesotho Government and UNDP Lesotho in October 2006 and it is available on line at http://www.undp.org.ls/energy/renewable_energy.php .

The Department of Energy of the Ministry of Natural Resources of Lesotho is implementing the Lesotho Renewable Energy-Based Rural Electrification (LREBRE) project in three districts: Mokhotlong, Thaba Tseka and Qacha’s Nek. The LREBRE project is a five years initiative aimed at improving people’s livelihoods by improving their access to affordable, clean energy services. It will also develop a sustainable PV market in the country starting with the three target districts. Its main global objective is to reduce Lesotho’s energy-related carbon dioxide emissions by introducing renewable energy technologies as substitute for fossil fuels in the rural areas remote from the national electricity grid. The project is funded by the Global Environment Facility (GEF) through the United Nations Development Programme (UNDP) and the Government of Lesotho.

The Project Document was signed between the Lesotho Government and UNDP Lesotho in October 2006 and it is available on line at http://www.undp.org.ls/energy/renewable_energy.php.

Six project outcomes are defined in the Project Document:

1. Delivery of renewable energy-based technology packages
2. Awareness raising on the potential role of renewable energy
3. Private and public sector working in the renewable energy sector strengthened and trained
4. Policy support and policy framework
5. Financial mechanisms for the larger scale dissemination of renewable energy
6. Learning and replication

Progress towards them is reported in Annual Project Implementation Reviews and other project progress reports including quarterly and annual reports (to be made available for the evaluator). Associated with these outcomes there are a number of Outputs (please see Appendix for the Revised Logical Framework of the project).

The Project is planned for five years (May 2007- May 2012).

Evaluation Audience and the Objectives of the Mid Term Evaluation

The Mid Term Evaluation (MTE) is an UNDP requirement for full size and medium size projects and it aims to provide managers with strategy and policy options for more effectively and efficiently achieving the project’s outcomes and for replicating the results. It also provides the basis for learning and accountability for managers and stakeholders.

The objective of this MTE is to measure the effectiveness and efficiency of project activities in relation to the stated objective so far, and to produce possible recommendations on how to improve the management of the project until its completion in 2012.

The MTE thus provides a valuable opportunity to assess signs of ultimate project success or failure and prompt necessary adjustments in project design and management. UNDP also views the MTE as an important opportunity to provide donors, government and project partners with an independent assessment of the status, relevance and performance of the project with reference to the Project Document.

Scope of the Evaluation

The evaluation should access, analyze, and provide recommendations under the following areas:

   1. Project concept and design

The evaluators will assess the project concept and design. He/she should review the problem addressed by the project and the project strategy, encompassing an assessment of the appropriateness of the objectives, planned outputs, activities and inputs as compared to cost-effective alternatives. The implementation approach and managerial arrangements should also be judged; as well as country ownership, stakeholder participation, and UNDP’s comparative advantage. The evaluator will assess the achievement of indicators and review the work plan, planned duration and budget of the project.

   2. Implementation

The evaluation will assess the implementation of the project in terms of quality and timeliness of inputs and efficiency and effectiveness of activities carried out. Also, the effectiveness of management as well as the quality and timeliness of monitoring and backstopping by all parties to the project should be evaluated. In particular, the evaluation is to assess the Project team’s use of adaptive management in project implementation. The following are key aspects of implementation that should be addressed:

a)      Monitoring Systems

• Assess the monitoring tools currently being used:
• Do they provide the necessary information?
• Do they involve key partners?
• Are they efficient?
• Are additional tools required?
• Reconstruct baseline data if necessary. Reconstruction should follow participatory processes and could be achieved in conjunction with a learning exercise;
• Ensure the monitoring system, including performance indicators, at least meets GEF minimum requirements. Apply SMART indicators as necessary;
  

b)      Risk Management

• Validate whether the risks identified in the project document and PIRs are the most important and whether the risk ratings applied are appropriate. If not, explain why. Describe any additional risks identified and suggest risk ratings and possible risk management strategies to be adopted;

c)      Work Planning

• Assess the use of the logical framework as a management tool during implementation and any changes made to it
• Ensure the logical framework meets UNDP/GEF requirements in terms of format and content
• Assess the use of routinely updated workplans;
• Are work planning processes result-based? If not, suggest ways to re-orientate work planning;
• Consider the financial management of the project, with specific reference to the cost-effectiveness of interventions. Any irregularities must be noted.

d)      Reporting

• Assess how adaptive management changes have been reported by the project management;
• Assess how lessons derived from the adaptive management process have been documented, shared with key partners and internalized by partners.

e)      UNDP Contribution

• Assess the role of UNDP. Consider:
• Field visits
• Steering Committee/TOR follow-up and analysis
• PIR preparation and follow-up
• GEF guidance
• Assess the contribution to the project from UNDP “soft” assistance (i.e. policy advice & dialogue, advocacy, and coordination). Suggest measures to strengthen UNDP’s soft assistance to the project management.

f)       Partnership Strategy

• Assess how partners are involved in the project’s adaptive management framework:
• Involving partners and stakeholders in the selection of indicators and other measures of performance
• Using already existing data and statistics
• Analyzing progress towards results and determining project strategies.
• Identify opportunities for stronger substantive partnerships;
• Assess how local stakeholders participate in project management and decision-making; Include an analysis of the strengths and weaknesses of the approach adopted by the project and suggestions for improvement if necessary;
• Consider the dissemination of project information to partners and stakeholders and if necessary suggest more appropriate mechanisms.

   3. Progress towards Results

a)      Measurement of change: Progress towards results should be based on a comparison of indicators before and after (so far) the project intervention. Progress can also be assessed by comparing conditions in the project site to conditions in similar unmanaged sites.
b)      Project strategy: how and why outcomes (listed as outputs in the project document) and strategies contribute to the achievement of the expected results. Examine their relevance and whether they provide the most effective route towards results.
c)      Sustainability: Extent to which the benefits of the project will continue, within or outside the project domain, after it has come to an end. Relevant factors include for example: development of a sustainability strategy, establishment of financial and economic instruments and mechanisms, mainstreaming project objectives into the local economy, etc.

Duties and Responsibilities

Product Expected from the Evaluation

The key product expected from this MTE is a comprehensive analytical report in English that should, at least, include the following contents:

1. Executive summary

• Brief description of the project
• Context and purpose of the evaluation
• Main conclusions, progress rating for project implementation and towards project objectives and outcomes (using the six-point UNDP/GEF rating scale), recommendations and lessons learned

2. Introduction

• Project background
• Purpose of the evaluation
• Key issues addressed
• The outputs of the evaluation and how will they be used
• Methodology of the evaluation
• Structure of the evaluation

3. The Project and its development context

• Project start and its duration
• Implementation status
• Problems that the project seek to address
• Immediate and development objectives of the project
• Main stakeholders
• Results expected

4. An analysis of the situation with regard to the outcomes, the outputs and the partnership strategy;

5. Key findings (including best practice and lessons learned, assessment of performance)

Project concept and design

• Implementation approach
• Country ownership/Driveness
• Stakeholder participation
• Replication approach
• Cost-effectiveness
• UNDP comparative advantage
• Linkages between project and other interventions within the sector
• Management arrangements

Implementation

• Monitoring systems
• Risk management
• Project management (work planning, reporting)
• UNDP country office contribution
• Partnership strategies

Results

• Measurement of results
• Project Strategy
• Prospects of sustainability
• Rating of project implementation (employing six-point UNDP/GEF Rating Scale)
• Rating of Progress towards project objectives and outcomes (employing the six-point UNDP/GEF rating scale.
• Tracking and verification of project co-financing.

6. Conclusions and recommendations

• Corrective actions for the design, implementation, monitoring and evaluation of the project
• Actions to strengthen or reinforce benefits from the project
• Proposals for future directions underlining main objectives
• Suggestions for strengthening ownership, management of potential risks

7. Lessons learned

• Good practices and lessons learned in addressing issues relating to effectiveness, efficiency and relevance.

8. Annexes:

TOR, itinerary, field visits, people interviewed, documents reviewed, etc.

The length of the mid-term evaluation report shall not exceed 30 pages in total (not including annexes).

Methodology or Evaluation Approach

An outline of an evaluation approach is provided below; however it should be made clear that the evaluation team is responsible for revising the approach as necessary. Any changes should be in-line with international criteria and professional norms and standards. They must be also cleared by UNDP before being applied by the evaluation team.

The evaluation must provide evidence-based information that is credible, reliable and useful. It must be easily understood by project partners and applicable to the remaining period of project duration.

The methodology to be used by the evaluation team should be presented in the report in detail. It shall include information on:

• Documentation review (desk study) - the list of documentation to be reviewed is included in the Appending B to this Terms of Reference;
• Interviews will be held with the following organizations and individuals at minimum: UNDP Lesotho, UNDP/GEF RTA from Pretoria, LREBRE project team, the Department of Energy, Project Steering Committee members, Project Manager, Project Director; PV installers, banks and beneficiary communities/households
• Field visits;
• Questionnaires;
• Participatory techniques and other approaches for the gathering and analysis of data.

Competencies

Evaluation Team

A team of independent experts will conduct the evaluation. The evaluators selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

The evaluation team will be composed of one International Consultant or team leader and one National Consultant. The consultants shall have prior experience in evaluating similar projects. Former cooperation with GEF is an advantage.

International consultant

1. Masters or Ph.D. degree in social sciences related to international development, i.e. economics, international relations, public and business administration or equivalent;

2. Extensive (at least 10-year) experience and proven track record with energy efficiency and/or renewable energy field, policy advice, development and implementation;

3. Highly knowledgeable of participatory monitoring and evaluation processes, and experience in evaluation of at least 3 projects with a major donor agencies;

4. Familiar with renewable energy technologies in Africa either through management and/or implementation or through consultancies in analysis and evaluation of energy efficiency and/or renewable energy projects;

5. Experience in credit schemes and other project financing schemes;

6. Demonstrated ability to assess complex situations, succinctly distills critical issues, and draw forward-looking conclusions and recommendations; 7. Ability and experience to lead multi disciplinary and national teams, and deliver quality reports within the given time; 8. Writing and communication will be in English, and must have excellent communication skills in English. The consultant must bring his/her own computing equipment.

Local consultant:

1. Masters degree in social sciences related to international development, i.e. economics, international relations, public and business administration or equivalent;

2. At least 5 years experience with energy efficiency and/or renewable energy field, policy advice, development and implementation;

3. Demonstrated skills and experience in development project implementation and management;

4. Knowledgeable on renewable energy institutions and projects in the country, climate change issues and priorities, and related policies and legislations in particular in relation to off-grid electrification;

5. Proficient in writing and communicating both in English and in Sesotho. Ability to interpret to the international counterpart and also to translate necessary written documents to English.

Team Qualities:

1. Recent experience with result-based management evaluation methodologies;

2. Experience applying participatory monitoring approaches;

3. Experience applying SMART indicators and reconstructing or validating baseline scenarios;

4. Recognized expertise in renewable energy technologies;

5. Familiarity with RE policies and management structures in Lesotho;

6. Demonstrable analytical skills;

7. Work experience in relevant areas for at least 10 years;

8. Experience with multilateral or bilateral supported projects;

9. Project evaluation experiences within United Nations system will be considered an asset;

10.Excellent English communication skills.


The National Consultant will provide input in reviewing all project documentation and will provide the International Consultant with a compilation of information prior to the evaluation mission. Specifically, the national expert will perform tasks with a focus on:

• Review documents;
• Prepare a list of the outputs achieved under project;
• Organize the mission programme and provide translation/interpretation when necessary;
• Participate in the design of the evaluation methodology;

Conduct an analysis of the outcome, outputs and partnership strategy (as per the scope of the evaluation described above);

Draft related parts of the evaluation report;

Assist Team leader in finalizing document through incorporating suggestions received on draft related to his/her assigned sections.

Required Skills and Experience

Individual consultants are invited to submit applications together with their CV for these positions. Or alternatively, proposals will be accepted from recognized consulting firms to field a complete team with the required expertise within the evaluation budget.

The evaluators must be independent from both the policy-making process and the delivery and management of assistance. Therefore applications will not be considered from evaluators who have had any direct involvement with the design or implementation of the project.

If selected, failure to make the above disclosures will be considered just grounds for immediate contract termination, without recompense. In such circumstances, all notes, reports and other documentation produced by the evaluator will be retained by UNDP.

If individual evaluators are selected, UNDP will appoint one Team Leader. The Team Leader will have overall responsibility for the delivery and quality of the evaluation products. Team roles and responsibilities will be reflected in the individual contracts. If a proposal is accepted from a consulting firm, the firm will be held responsible for the delivery and quality of the evaluation products and therefore has responsibility for team management arrangements.

Implementation Arrangements

Management Arrangements

The principal responsibility for managing this evaluation lies with UNDP Lesotho. UNDP Lesotho will contract the evaluators and ensure the timely provision of per diems and travel arrangements within the country for the evaluation team. UNDP Lesotho and the Project Team will be responsible for liaising with the Evaluators team to set up stakeholder interviews, arrange field visits, coordinate with the Government etc.

Timeframe and resources required

The duration of the review will be 10 days for desk review, interviews, and field trips; and 5 days for report writing and submission of the draft report.  It is intended to commence during or around the first two weeks of February 2011.

The activities should be as follows:

• desk review
• briefings for evaluators
• visits to the field (including allocation for travel), interviews, questionnaires debriefings
• validation of preliminary findings with stakeholders through circulation of initial reports for comments, meetings, and other types of feedback mechanisms
• preparation of final evaluation report (length, format and content should be specified)

Prior to approval of the final report, a draft version shall be circulated for comments to government counterparts, project team and UNDP Country Office (CO) and Regional Coordination Unit (RCU) and shall be thoroughly discussed with the evaluator to clear any misunderstandings or misinterpretations. If subsequent to any discrepancies have emerged between impressions and findings of the evaluation team and the aforementioned parties, these should be explained in an annex attached to the final report.

The proposed dates for the in-country mission to Lesotho are 9th January 2011. The assignment is to commence no later than the third week of January 2011.

To apply for his position, please use the following website: http://jobs.undp.org/. Interested parties are expected to upload a copy of their Curriculum Vitae. The deadline for the receipt of applications is January 9th 2011. Regrettably, only short-listed candidates can be notified.