Antecedentes

BSEEP has for its goal the reduction in the annual growth rate of GHG emissions from the Malaysia buildings sector.  The project objective is the improvement of the energy utilization efficiency in Malaysian buildings, particularly those in the commercial and government sectors, by promoting the energy conserving design of new buildings and by improving the energy utilization efficiency in the operation of existing buildings. The realization of this objective will be facilitated through the removal of barriers to the uptake of building energy efficiency technologies, systems, and practices. The project is in line with the GEF’s climate change strategic program on Promoting Energy Efficiency in Residential and Commercial Buildings (SP-1). It is comprised of activities aimed at improving energy efficiency and promoting the widespread adoption of energy efficient building technologies and practices in the Malaysian buildings sector.
 
Specifically, the proposed project will reduce carbon emissions by an estimated 581.1 ktons.
CO2 per year (or cumulative total of about 1,421.3 ktons CO2) by end of the project. This represents about 4% reduction in CO2 emissions compared to the magnitude of CO2 emissions under a business-as-usual scenario13. Five years after the project end, CO2 emissions are forecast to be about 7.2% lower in annual emissions if there will be no BSEEP.

The expected outcomes of the project are the following:
  • Clear and effective system of monitoring and improving the energy performance of the building sector;
  • Implementation of, and compliance to, favorable policies that encourage the application of EE technologies in the country’s buildings sector;
  • Availability of financial and institutional support for initiatives on EE building technology applications;
  • Enhanced awareness of the government, public and the buildings sector on EE building technology applications;
  • Improved confidence in the feasibility, performance, energy, environmental and economic benefits of EE building technology applications leading to the replication of the EE technology application demonstrations.
 Outcomes and Outputs of the BSEEP are as in Annex 4. Please download Annex 4 from this link: http://www.undp.org.my/files/editor_files/files/MyIC_2013_010%20%26%20011_%20TOR%20for%20BSEEP%20with%20Annexes.pdf
 
BSEEP is Nationally-Executed (NEX) by the Malaysian Government and JKR is the appointed executing agency.
The Project Document and other relevant GEF documents can be downloaded from the following weblink:  http://www.thegef.org/gef/sites/thegef.org/files/repository/11-30-09%20ID3598%20-%20Council%20letter.pdf
Information on the UNDP evaluation process and experience from other countries  can be refered to the Evaluation Resource Center  at the following weblink http://erc.undp.org.  
Information on project can be viewed at www.jkr.gov.my/bseep.
 
Objective of the Mid Term Review

The objective of this MTR is to gain an independent analysis of the progress of the project so far.  The MTR will identify potential project design problems, assess progress towards the achievement of the project objective, identify and document lessons learned (including lessons that might improve design and implementation of other UNDP-GEF projects), and make recommendations regarding specific actions that should be taken to improve the project.  The MTR will assess early signs of project success or failure and identify the necessary changes to be made. The project performance will be measured based on the indicators of the project’s logical framework (see Annex 4) and various Tracking Tools.
 
Specifically, the purpose of the BSEEP Mid-Term Review is to review, rate the performance of the project from the start of the project implementation up to the present and recommend possible corrective actions (in short and long-term actions). The review will include evaluating the:
  • Progress in project implementation, measured against planned outputs set forth in the Project Document/Inception Report with latest revision in accordance with rational budget allocation;
  • An assessment of the overall impact of the project to the country; and
  • Identify corrective measures, lessons learned and best practices for immediate actions.
The MTR must provide evidence based information that is credible, reliable and useful.  The review team is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, in particular the GEF operational focal point, UNDP Country Office, project team, UNDP GEF Technical Adviser based in the region and key stakeholders. The team will review all relevant sources of information, such as the project document, project reports – including Annual APR/PIR, project budget revisions, progress reports, GEF focal area tracking tools, project files, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based review. A list of documents that the project team and UNDP Country Office will provide to the team for review is included under section 5 of this Terms of Reference.

Deberes y responsabilidades

The consultant will be responsible for the delivery, content, technical quality and accuracy of the evaluation, as well as the recommendations. The consultant will not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

The scope of work for the consultancy will include, but not limited to, the following activities:
 
Assessment of progress in project implementation:

The MTR will focus on such aspects as appropriateness and relevance of work plan, compliance with the work plan alongside with budget allocation; timeliness of disbursements; procurement, quantity and quality of goods and services created; coordination among different project stakeholders. Any issue that has impeded the implementation of the project or any of its components, including actions taken and resolutions made should be highlighted.   Activities that should be taken up by the NPT and  the executing agency shall also be recommended.  The template below shall assist the consultant in reviewing the progress.

Review of Activities:
  • Component/Activities;
  • Planned;
  • Actual;
  • Action.

Assessment of Budget Utilization:

  • Component/Activities;
  • As per ProDoc;
  • Actual Expenditures;
  • % of Project Budget.
The following assessments shall be carried out:
  • Project design (as per the LFA), i.e., whether the project design allowed for flexibility in responding to internal and external changes in the project environment;
  • Implementation difficulties, i.e., whether difficulties and barriers, which were not expected at the start of the project, are identified and the approaches for the solutions are considered and implemented effectively;
  • Project resources, i.e., whether the project components and activities were logically designed as to content and time frame commensurate with the human and financial resources that were made available;
  • The review team will assess the following project progress.  For each category, the review team is required to rate overall progress using a six-point rating scale outlined in Annex 1.
Assessment of project outputs: 

General questions will include the below:

  • Whether the project is implemented in the right direction to achieve the outcomes (i.e., based on the agreed work plan / annual target);
  • Review how the project addresses country priorities and the significance of the outcomes so far achieved for the country/region;
  • Whether the project outputs are produced effectively, efficiently, and in a timely manner according to the time schedule;
  • The quality and credibility of the outputs, as stipulated in the Project Document;
  • How effective and efficient the project funds are utilized, and how the expenditures are monitored;
  • The credibility of the data used in the project and reliance of the numerical outputs;
  • The monitoring and evaluation of the project consultants’ work;
  • The quality of the internal monitoring system results;
  • Review the extent to which the implementation of the project has been inclusive of relevant stakeholders and to which it has been able to create collaboration between different partners.

On the overall project level, it will assess the project performance in terms of:

  • Progress towards achievement of results;
  • Factors affecting successful implementation and achievement of results;
  • Project Management framework; and
  • Strategic partnerships.

Progress towards achievement of results (internal and within project’s control):

  • Review the problem addressed by the project and the underlying assumptions.  Review the effect of any incorrect assumptions made by the project.  Identify new assumptions;
  • Review the relevance of the project strategy and assess whether it provides the most effective route towards results;
  • Assess the outputs and progress toward outcomes achieved so far and the contribution to attaining the overall objective of the project;  
  • Is the Project making satisfactory progress in achieving project outputs vis-à-vis the targets and related delivery of inputs and activities?
  • Are the direct partners and project consultants able to provide necessary inputs or achieve results?
  • Given the level of achievement of outputs and related inputs and activities to date, is the Project likely to achieve its Immediate Purpose and Development Objectives?
  • Are there critical issues relating to achievement of project results that have been pending and need immediate attention in the next period of implementation?
  • Examine if progress so far has led to, or could in the future lead to, beneficial development effects (i.e. income generation, gender equality and women’s empowerment, improved governance etc...) that should be included in the project results framework and monitored on an annual basis;
  • Examine whether progress so far has led to, or could in the future lead to, potentially adverse environmental and/or social impacts/risks that could threaten the sustainability of the project outcomes.  Are these risks being managed, mitigated, minimized or offset?  Suggest mitigation measures as needed.
Factors affecting successful implementation and achievement of results (beyond the Project’s immediate control or project-design factors that influence outcomes and results):
  • Is the project implementation and achievement of results proceeding well and according to plan, or are there any outstanding issues, obstacles, bottlenecks, etc. on the consumer, government or private sector or the energy efficient buildings industry as a whole that are affecting the successful implementation and achievement of project results?
  • Review the baseline data included in the project results framework and GEF Tracking tool and suggest revisions as necessary;
  • Is the project logical framework and design still relevant in the light of the project experience to date?
  • Is the project well-placed and integrated within the national government development strategies, such as community development, poverty reduction, etc., and related global development programs to which the project implementation should align?
  • Do the Project’s purpose and objectives remain valid and relevant, or are there items or components in the project design that need to be reviewed and updated?
  • Are the Project’s institutional and implementation arrangements still relevant and helpful in the achievement of the Project’s objectives, or are there any institutional concerns that hinder the Project’s implementation and progress.

Project management (adaptive management framework):

  • Are work planning processes result-based?  If not, suggest ways to re-orientate work planning to focus on results.
  • Are the project management arrangements adequate and appropriate?
  • How effectively is the project managed at all levels? Is it results-based and innovative?
  • Do the project management systems, including progress reporting, administrative and financial systems and monitoring and evaluation system, operate as  effective management tools, aid in effective implementation and provide sufficient basis for evaluating performance and decision making?
  • Review the monitoring tools currently being used:  Do they provide the necessary information? Do they involve key partners? Do they use existing information? Are they efficient? Are they cost-effective? Are additional tools required?
  • Ensure that the monitoring system, including performance indicators, meet GEF minimum requirements.  Apply SMART indicators as necessary;
  • Ensure broader development and gender aspects of the project are being monitored effectively.  Develop SMART indicators, including disaggregated gender indicators as necessary;
  • Review the mid-term GEF Tracking Tool (s) as appropriate and comment on progress made, quality of the submission, and overall value of the GEF Tracking Tool;
  • Examine the financial management of the project monitoring and evaluation budget.  Are sufficient resources being allocated to M&E? Are these resources being allocated effectively?
  • Is technical assistance and support from project partners and stakeholders appropriate, adequate and timely?
  • Validate whether the risks originally identified in the project document APR/PIRs and the ATLAS Risk Management Module still hold and are the most critical. Assess whether risk ratings applied are appropriate. If not, explain why?
  • Describe additional risks identified and suggest risk ratings and possible risk management strategies to be adopted;
  • Assess the use of the project logical framework and work plans as management tools and and review any changes made to it since project start. Ensure any revisions meet with UNDP-GEF requirements in planning and reporting and assess the impact of the revised approach on project management?
  • Consider the financial management of the project, assess the cost effectiveness of the interventions and note any irregularities;
  • Complete the co-financing monitoring table (see Annex 3);
  • Review the changes to fund allocations as a result of budget revisions and assess the appropriateness and relevance of such revisions;
  • How have the APR/PIR process helped in monitoring and evaluating the project implementation and achievement of results? 
  • Assess how adaptive management changes have been reported by the project management, and shared with the Project Board;
  • Assess how lessons derived from the adaptive management process have been documented, shared with key partners and internalized by partners;
  • Review overall effectiveness of project management as outlined in the project document.  Have changes been made and are they effective?  Are responsibilities and reporting lines clear?  Is decision-making transparent and undertaken in a timely manner?  Recommend areas for improvement;
  • Review the quality of execution of the project Implementing Partners and recommend areas for improvement;
  • Review the quality of support provided by UNDP and recommend areas for improvement.

Strategic partnerships (project positioning and leveraging):

  • Are the project partners and their other similar engagements in the project strategically and optimally positioned and effectively leveraged to achieve maximum effect of the EE program objectives for the country?
  • Asses how project partners, stakeholders and co-financing institutions are involved in the Project’s adaptive management framework;
  • Identify opportunities for stronger collaboration and substantive partnerships to enhance the project’s achievement of results and outcomes;
    Are the project information and progress of activities disseminated to project partners and stakeholders? Are there areas to improve in the collaboration and partnership mechanisms?
Deliverables and Timeline:

The MTR Team will consist of one International Consultant and one National Consultant. The International Consultant will be the Team Leader. The Team Leader, in close collaboration with the National Consultant, will have the overall responsibility for the quality and timely submission of the deliverables. Specifically, the team of consultants is responsible for submitting the following deliverables to the UNDP Country Office and Project Support Unit:
  •  Inception Report or Submission of Work plan. To review team clarifies timing and method of review within 5 days of signing of the contract Review team submits to UNDP Country Office;
  • Presentation Initial Findings at the end of review mission to project management, UNDP Country Office, reviewed by RTA;
  • Draft Final Report Full report (as template in annex 2) with annexes. A detailed record of consultations with stakeholders will need to be kept and provided (as part of the information gathered by the reviewers), as an annex to the main report. If there are any significant discrepancies between the impressions and findings of the review team and stakeholders, these should be explained in an annex attached to the final report. Duration: within 2 weeks of the review mission and sent to UNDP CO, reviewed by RTA, PCU, GEF OFP;
  • Final Report. A Revised report with audit trail detailing how all received comment have (and have not) been addressed in the final review report) within 2 weeks of receiving UNDP comments on draft and sent to UNDP CO.
 Methodology:

The reviewers will review relevant project documents and reports related and conduct focused individual/group discussions on topics and issues that relate to the implementation and impact of the project. The reviewers are expected to become well versed as to the objectives, historical developments, institutional and management mechanisms and project activities. More specifically, the review will be based on the following sources of information:
  • Review of documents related to the project such as project document, quarterly and annual progress reports, other activity/component specific reports and evaluation  as described;
  • Structured interview with knowledgeable parties, i.e., NPD, Project Staff members, Sub-Contractors, International/National Consultants, UNDP, members of the National Steering/Advisory Committee/s, Project Beneficiaries or grantees, etc;
  • Site visits to specific projects, if feasible. The site visits should be discussed with the NPM and the UNDP Country Office.
Key documents to be reviewed are as below:
  • Project Initial Form (PIF);
  • UNDP/GEF BSEEP Project Document;
  • Inception Report;
  • All output reports and documents produced under BSEEP;
  • Minutes of Project Steering Committee Meetings and National Steering Committee meetings;
  • Amendments to the inception report (if any);
  • Review/evaluation report;        
  • Latest Project Implementation Report PIR;
  • Latest NEX audit reports or any other audit reports;
  • Past consultancies’ assignments and summary of the results;
  • Quarterly reports;
  • Pictures of equipment, installations and sites if any;
  • Newspaper/publication articles.
The evaluation team shall meet and interview the following:
  • National Project Director;
  • National Project Manager;
  • Finance Assistant;
  • Project Executive;
  • Component Managers (all) if any;
  • Key government stakeholders in building energy efficiency (i.e. EPU International Cooperation, EPU Energy, MEGTW, Energy Commission, Sustainable Energy Development Authority (SEDA);
  • Other sections/departments in the Implementing Partner relevant to BSEEP;
  • Representative from the Industry association;
  • Representative from the academia relevant to BSEEP;
  • Selected members of the NSC meeting;
  • Consultants;
  • Participating industries / demonstration sites;
  • Other project partners relevant to the outcome of the project.
The evaluator will conduct an opening meeting with the NPD and relevant executing agency staff to be followed by an “exit” interview with UNDP CO to discuss the findings of the assessment prior to the submission of the final report. 
 
Duration:

The total duration of the MTR will be 4 weeks in August or September 2013. The expected number of working days per consultant is 20 days including 10 working days mission to Malaysia for the international consultant.  Detail dates will be confirmed later.
 
Implementation Arrangements:

The principal responsibility for managing this review resides with the UNDP Country Office (UNDP CO) in Kuala Lumpur, Malaysia.  The UNDP CO will contract the consultants and ensure the timely provision of per diems and travel arrangements within the country for the review team.  The BSEEP project team will be responsible for liaising with the review team to set up stakeholder interviews, arrange field visits with missions to relevant project sites.
 
In preparation for the review mission, the project manager, with assistance from UNDP country office, will arrange for the completion of the tracking tools. The tracking tools will be completed/endorsed by the relevant implementing agency or qualified national research /scientific institution, and not by the international consultant or UNDP staff. The tracking tools will be submitted to the mid-term review team for comment.  These comments will be addressed by the project team, and the final version of the Tracking tools will be attached as annexes to the Mid-term evaluation report.
 
Terms of Payment:
  • Inception Report / workplan – 10%;
  • Initial finding presentations – 20%;
  • Draft Final Report – 30%;
  • Final Report – 40%.
Application Process:
 
Documents to be included when submitting the proposal
Interested individual consultants must submit the following documents/information to demonstrate their qualifications: 
All the documents above must be attached and uploaded as one PDF document.
Incomplete applications will be excluded from further consideration.
 
Any request for clarification must be send in writing to procurement.my@undp.org. Please write the Project reference "MyIC/2013/010" in your e-mail heading.

The deadline for submitting requests for clarification/questions is 15 July 2013.
 
Proposal/letter can be submitted to:

United Nations Development Programme
Wisma UN, Kompleks Pejabat Damansara
Jalan Dungun, Damansara Heights
50490 Kuala Lumpur, Malaysia. 
 
Criteria for Evaluation of Proposal:  The selection will be made based on the educational background and experience on similar assignments.  It will be a 70:30 combined weight scoring where the financial proposal will weigh as 30% of the total scoring.
 

Competencias

Core competencies
  • Demonstrates commitment to UNDP’s mission, vision and values;
  • Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability;
  • Highest standards of integrity, discretion and loyalty.
Functional competencies
  • Results orientated and accountability;
  • Capacity in planning and organizing;
  • Communication and trust; 
  • Client orientation;
  • Organizational development and innovation;
  • Competence in project Adaptive Management;
  • Demonstrable analytical skills.

Habilidades y experiencia requeridas

Education:
  • Tertiary education in science, engineering or in relevant climate change mitigation.  Post-graduate or with relevant professional qualification is preferred;

Experience:

  • More than 5 years of working experience in the  national level capacity building projects, project management and evaluation, climate change management and with a good knowledge of the state-of-the-art approaches and international best practices;
  • Prior knowledge of GEF and UNDP reporting frameworks, GEF principles and expected impacts in terms of global benefits, and the policy, legal and institutional environment of Malaysia would be an advantage;
  • Experience with UN / UNDP / GEF result-based management evaluation methodologies. Project evaluation experiences within United Nations system will be considered an asset;
  • Experience applying SMART indicators and reconstructing or validating baseline scenarios;
  • Experience in reviewing or evaluating similar projects will be an advantage;
  • Experience with GEF financed projects is an advantage;
  • Familiar with project management and financial framework including output/outcome and impact analysis;
  • Familiar with the national institutional framework and local set-up of the project.

Language:

  • Well versed with both English and Bahasa Malaysia, understands institutional framework and able to facilitate discussions among the local stakeholders.

FC: 62000