Background

UNDP Bangladesh Country Programme Document (CPD) 2012-2016[1] responds to the Government’s development priorities and vision of attaining middle income country status by 2021, as stated in the Outline Perspective Plan 2010-2021 and ongoing Sixth Five Year Plan (2011-2015). The present CPD focuses on three areas: (a) democratic governance and human rights; (b) pro-poor growth with equity; and (c) climate change, environment, disaster risk reduction and response.

In line with the Country Programme Evaluation Plan 2012-2016 agreed with the Economic Relation Division of the Ministry of finance, a mid-term outcome evaluation is carried out to assess UNDP’s support towards achieving outcomes 3.1 and 3.2:

• By 2016, populations vulnerable to climate change and natural disaster have become more resilient to adapt to risks (Country Programme Outcome 3.1/UNDAF Outcome 5.1);
• By 2016, vulnerable populations benefit from better natural resource management and access to low carbon energy (Country Programme Outcome 3.2/UNDAF Outcome 5.2).

To the present, Bangladesh remains off-track to meet MDG 7 (environment) with sub-par performance due to deforestation, growth of urban slums, biodiversity loss and pollution. About half of the population is dependent on a rapidly degrading natural resource base, including forests, for their livelihoods and only about 50 per cent of households have access to on-grid energy. Bangladesh’s latest MDG report noted several key challenges, such as efficiently using forest resources, developing water-efficient agricultural practices and establishing proper policies and regulation frameworks. These environmental threats, compounded by Bangladesh’s population pressure and institutional capacity constraints, can undermine its development potential as well as reverse recent hard-earned gains.

The geography and topography of Bangladesh make it one of the most vulnerable countries in the world to natural disasters, which have increased in frequency and severity as a result of the worsening impact of climate change. The poorest are the most severely affected as their adaptive capacity is relative lower. While better disaster preparedness strategies and practices have, over time, reduced the number of deaths, the loss of assets and livelihoods continues to increase (as much as 3 per cent of GDP), with women being most acutely affected. Climate change adaptation (CCA) and disaster risk reduction (DRR) are thus critical areas of concern for Bangladesh. UNDP programmes, current and past, continues to establish strong building blocks that would go to scale alongside wider adaptation and mitigation efforts in line with the Bangladesh Climate Change Strategy and Action Plan. Under the CPD Outcome 3.1 and 3.2, UNDP is focusing on developing the capacity of the Government to mainstream the climate-environment-poverty nexus into policy and planning frameworks across ministries, while augmenting community-based risk reduction and adaptation capacities. By 2013, Bangladesh has achieved notable developments in these outcome areas, including the development of an environment, climate and disaster vulnerability index, which has the capacity to measure the vulnerability and ability of people living in 19 coastal districts to cope with climate change and disaster. Also, Bangladesh has made major progress in promoting green development with policies that take into consideration the relationship between its national contexts and globally accepted frameworks and protocols. UNDP’s long-term advocacy and technical support to the Department of Environment, Ministry of Environment and Forest and Ministry of Industry were critical to achieving many policy successes.

The estimated resource envelope for the UNDP Bangladesh Country Programme 2012-2016 is US$ 553.6 million, where indicative budgets of the Outcome 3.1 and 3.2 comprise 25.9% (US$ 143.6 million) of the overall CPD envelop. Till July 2014, the Outcome 3.1 and 3.2 altogether have mobilized 78.0% (US$ 112 million) of its planned budget from UNDP core resources as well as from its development partners.

Duties and Responsibilities

Objective:

Evaluation Purpose

Purpose: The purpose of this mid-term outcome evaluation is to take stock of achievements to date, document lessons learned and propose ways to correct the course of intervention for the next two years of the country programme cycle.

TIMING: The mid-term evaluation is conducted in the second half of 2014 because it is the penultimate year to the MDG deadline, and the midway point of the 2012–2016 CPD/UNDAF. The timing is in line with the CPD/UNDAF Evaluation Plan 2012-2016, which foresees that UNDP Bangladesh undertakes a mid-term outcome evaluation on the CPD Outcome 3.1 and 3.2 in 2014. The timing of the evaluation also ensures that evaluation results will be key input into the UNDAF mid-term review scheduled for early 2015.

Utilization: The evaluation results are to be utilized by not only UNDP Bangladesh but also by the government partners as well as other key development partners/donors. They will also become critical inputs to the UNDAF mid-term review scheduled in early 2015, where UNDP acts as lead agency in four UNDAF pillars and these outcomes contribute directly to UNDAF Pillar 5.  The mid-term outcome evaluation also aims to identify which UNDP approaches have worked well and which have faced challenges, use lessons learned to improve future initiatives and generate knowledge for wider use at all levels (corporate, regional and country) and serve as a means of quality assurance and hold UNDP accountable for the resources invested in its work as well as for its national partners and stakeholders. Notably, the evaluation will become critical inputs for the alignment of UNDP’s new Strategic Plan (SP) 2014-2017[1]. Following the evaluation conclusions and recommendations pertinent to UNDP, UNDP Bangladesh will prepare a management response and implement follow-up actions through the UNDP Evaluation Resource Centre (ERC).

Scope of work:

Evaluation Objectives and Scope

Objectives

The mid-term outcome evaluation will primarily assist UNDP Bangladesh in assessing its effectiveness in progressing towards the outcomes. The evaluation will assess if populations vulnerable to climate change and natural disasters are now more resilient to adapt to risks compared to before 2012. The evaluation will also assess to what extent vulnerable populations now benefit from better natural resource management and access to low carbon energy compared to before 2012. This includes an assessment on the causal linkage if and/or by which outputs contribute to the achievement of the outcomes and the extent to which the planned outcome has been achieved and likely to be achieved by the end of 2016. It will assess both negative and positive factors that facilitate and/or hinder the progress in achieving the outcome including the external environment, weaknesses in design, management and resources. In addition, the evaluation will assess the relevance, efficiency, effectiveness and potential sustainability of UNDP’s interventions on resilience to climate change and natural disasters and on the benefit from natural resource management and access to low carbon energy.

Scope

The mid-term outcome evaluation covers a time span from January 2012 - the beginning of the CPD cycle - up to the outset of the evaluation. While the programmes/projects which are at policy level and/or with global compliance has a nation-wide coverage, project sites include socio-economically backward and disaster vulnerable geographic areas (cyclone, flood and drought-prone) districts for the field visit and data collection will be identified and selected given the available time for site visits and in consultation with the Climate Change, Environment and Disaster (CCED) Cluster of UNDP Bangladesh. While there have been 18 programmes/projects implemented under the outcome 3.1 and 3.2 during the CPD cycle, this evaluation focuses on key programmes and projects which contributed directly to achieving the outcomes. This includes projects and programmes that started prior to 2012 but continued into 2012 and beyond. The primary stakeholders include government institutes directly involved with project implementation: Ministry of Disaster Management & Relief, Ministry of Environment & Forests, Ministry of Power & Mineral Resources, Ministry of Agriculture, Ministry of Water Resources, Ministry of Land, Ministry of Fisheries and Livestock, Ministry of Industries, Forest Department in the Chittagong Hill Tracts (CHT). Other constituencies include private brick and pharmaceutical sectors, civil society organizations, academia and direct beneficiaries such as ethnic minorities.

Evaluation Questions

The following evaluation questions help define the information that the evaluation will generate. The evaluation questions are formulated and clustered to address the primary evaluation criteria[1] and the effectiveness criteria will be weighted most from other criteria in this evaluation to demonstrate how UNDP initiatives have or have not contributed to the achievement of the outcome. The evaluation questions may be adjusted upon consultations and discussions with the programme cluster during the inception phase.

Country Programme Outcome 3.1/UNDAF Outcome 5.1.

By 2016, populations vulnerable to climate change and natural disaster have become more resilient to adapt to risks

The outcome formulation contains different elements. That is why the outcome formulation needs to answer evaluation questions from at least two perspectives:

• Resilience  related to risks from climate change;
• Resilience  related to risks from natural disaster.

Relevance of UNDP's involvement and approach:

• To what extent are UNDP's key programmes/projects relevant to make vulnerable populations more resilient to the risks of climate change and natural disasters?

Effectiveness in contributing to the achievement of the outcome:

• Compared to 2011, are vulnerable populations now more resilient against climate change?
• Compared to 2011, are vulnerable populations now more resilient against natural disasters?
• To what extent are these changes due to UNDP's interventions?

Efficiency in delivering outputs:

• To what extent were the relevant programme/project outputs delivered in time and in good quality;
• To what extent did UNDP ensure value for money?
• Has there been any duplication of efforts among UNDP’s own interventions and interventions delivered by other organizations or entities in contributing to the outcome?

Sustainability of the outcome:

• What indications are there that the outcome will be sustained after external funding ends?

 Country Programme Outcome 3.2/UNDAF Outcome 5.2

By 2016, vulnerable populations benefit from better natural resource management and access to low carbon energy

The outcome formulation contains different elements. That is why the outcome formulation needs to answer evaluation questions from at least two perspectives:

• Better natural resource management;
• Better access to low carbon energy.

Relevance of UNDP's involvement and approach:

• To what extent are UNDP's key programmes/projects relevant to increase benefits from better natural resource management and access to low carbon energy?

Effectiveness in contributing to the achievement of the outcome:

• Compared to 2011, to what extent do vulnerable populations now benefit more from better natural resource management;
• Compared to 2011, to what extent do vulnerable populations now benefit from better access to low carbon energy?
• To what extent are these changes due to UNDP's interventions?

 Efficiency in delivering outputs:

• To what extent were the relevant programme/project outputs delivered in time and in good quality?
• To what extent did UNDP ensure value for money?
• Has there been any duplication of efforts among UNDP’s own interventions and interventions delivered by other organizations or entities in contributing to the outcome?

 Sustainability of the outcome:

• What indications are there that the outcome will be sustained after external funding ends?

In addition, the evaluation should also consider if and how UNDP promoted gender equality, human rights and human development in the delivery of outputs.

Methodology

The suggested approach and method for conducting this evaluation is to use a multi-level, mixed-method approach[1]. This includes collecting both quantitative and qualitative data sets on multiple levels that will need to be validated and triangulated. The overall guidance on evaluation methodologies is found in the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results.

It is for the evaluation team to examine and determine the specific design and methods for this mid-term outcome evaluation during the initial inception period in close consultation with the programme Cluster. During the inception phase, the evaluation team will elaborate a detailed methodology how to answer each evaluation question.

The following data collection methods and analysis should be applied, but not limited to:

• Desk reviews of relevant documents (CPD, UNDAF, relevant project documents, reports of relevant flagship projects, project surveys, studies relating to the country context and situation, evaluation reports, etc.);
• Key informant interviews and/or focus group discussions with beneficiaries or representatives of beneficiaries, government partners both at the central and local levels, development partners/donors including bilateral and multilateral partners (AusAid, DFID, European Union, Norwegian Embassy, Sida), other UN agencies, NGOs, and CSOs working to contribute to the same outcome; UNDP Country Office’s senior management, programme, and project staff, etc. The selection of interview partners should follow a deliberate purposive sampling strategy;
• Direct observations during visits to national implementing  partners and field visits;
• Gap analysis and review of national statistics as well as administrative and survey data collected by the programmes/projects and other stakeholders, including the CPD outcomeindicators and the new corporate Strategic Plan[1] (SP) 2014-2017 indicators, data and information from the 4W Database, the DMIC Portal, Cyclone shelter DB, Union Fact sheets, Inundation Maps, CDMP Risk Management Information; review of the Profitability & Replication Study on HHK Demo Plants, UNDP 2013. The evaluation team is expected to review existing data sets from programmes/projects and the national statistics relevant to this outcome evaluation, and identify what/where the gaps exist for the CO in aggregating the data and monitoring the CPD and the Strategic Plan indicators for the rest of the monitoring cycles (till 2016 for the CPD and 2017 for the Strategic Plan);
• International indices and databases on resilience in case of climate change and natural disasters in Bangladesh;
• Quantitative analysis of budgets and expenditure reports in ATLAS and project reports;
• Administration of surveys or questionnaires, as applicable.

The data collection methods should be participatory and inclusive of disadvantaged and marginalized populations. Major methodological limitations or limitations based on data collections should be noted in the final evaluation report.

Evaluation Team Composition and Required Competencies

The evaluation team will comprise 4 members: one team leader, two evaluation experts and one data analyst. The presence of an international consultant is deemed desirable given the complexity and sensitivity of some of the issues concerned, and therefore to safeguard independence and impartiality of the evaluation.

Supervision and performance evaluation:

Implementation Arrangement

This evaluation is commissioned by UNDP Bangladesh. The Climate Change, Environment Disaster (CCED) Cluster in the Country Office will be responsible for coordinating and managing the evaluation throughout the entire process and provide the evaluation team any logistics and administrative support as needed. The Results and Resource Management Cluster (RRMC) will serve as quality assurance to provide overall technical guidance and ensure the corporate compliance of outcome evaluations. The consultant will be under direct supervision of the head of the CCED Cluster in close consultation with the RRMC. In order to guide the evaluation process and assure quality, an evaluation reference group is planned to be formed from approximately five members from the Cluster, the Country Office, key stakeholders, the Regional Bureau for Asia and the Pacific or the Independent Evaluation Office, where the members are asked to provide inputs on the ToR, selection of consultants, inception report, draft report and the final report.

Evaluation Ethics

Evaluations in UNDP will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’[1]. Evaluators should ensure to be in compliance with evaluation ethics and procedures to safeguard the rights and confidentiality of information providers during the designing, implementing and managing evaluation activities.

Timeframe and deadlines:

This evaluation assignment is contracted for a total period of 40 days and should be completed by December 7, 2014. The following activities should commence within four weeks of signing the contract:

• Comprehensive literature review and analysis of background data including project documents, project/Country Office/UNDAF progress reports, annual reviews, evaluation reports and other key documents (Home based -05days);
• Briefing and kick-off meeting with UNDP staff from respective Cluster(s) and RRMC (Dhaka-01 day);
• Prepare and submit the inception report including the adjusted work plan, an evaluation matrix and other items as in the Companion Guide(Dhaka -04 days);
• Conduct data collection and analysis, interviews, site visits, stakeholder meetings, workshops, etc. in the country (Dhaka and outside of Dhaka -15 days);
• Prepare and submit the draft evaluation report to UNDP (Dhaka-07 Days);
• Debrief with key stakeholders and present key findings and recommendations; collect feedback from the debriefing workshop (Dhaka-01 Day);
• Incorporate comments from key stakeholders, respective Cluster(s) and the government partners (Dhaka -04 Days;
• Finalize and submit the final evaluation report and evaluation brief to UNDP (Dhaka-03 Days).

 Expected outputs/deliverables:

The consultant, working closely with other evaluation team members, is responsible for submitting the following deliverables to UNDP Bangladesh at the agreed work plan:

• Inception Report: An inception report should be prepared by the evaluators before going into the full-fledged data collection exercise. It should detail the evaluators’ understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods, proposed sources of data and data collection procedures. The inception report should elaborate and finalize the two evaluation matrices for outcome 3.1 and 3.2 and a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product. The inception report provides the programme Cluster and the evaluators with an opportunity to verify that they share the same understanding about the evaluation and clarify any misunderstanding at the outset. The inception report should follow the structure proposed in the UNDP Outcome-Level Evaluation: A Companion Guide, Annex 1, p.31[1];
• Draft Evaluation Report: The evaluation report should follow the structure outlined in the UNDP Outcome-Level Evaluation: A Companion Guide, p. 29-30. The programme Cluster and key stakeholders in the evaluation should review the draft evaluation report to ensure that the evaluation answers in depth all evaluation questions and backs up the arguments with credible and sufficient quantitative and qualitative evidence. The draft evaluation report should not exceed 40 pages without annexes;
• Presentation: Debriefing with stakeholders to present key findings and recommendations and collect feedbacks
• Final Evaluation Report: The final report will reflect the comments and feedback from stakeholders, including feedback provided during the presentation;
• Evaluation Brief: A concise summary of the evaluation findings in plain language that can be widely circulated. This can be in a form of a PowerPoint presentation or a two-page briefing document;
• Data Review Report:  The report is a supplementary document to the final evaluation report describing, but not limited to, 1) an analysis and findings on data gaps and availability of independently verifiable sources in monitoring the CPD and the Strategic Plan indicators among existing national statistics, administrative and project data sets relevant to this CPD outcome  evaluation and 2) recommendations on how to maximize the use of existing project data and how to harmonize or fill in the gaps of data and information between now and 2016/2017 (the CPD and the Strategic Plan respectively). The report should not exceed 10 pages without annexes.

Inputs

UNDP Bangladesh will provide office space and transport to field sites and will also arrange meetings and consultations to ensure access to key stakeholders. The CCED Cluster will provide key documents and data relevant to the evaluation. No laptop will be provided.

 Tentative Payment Schedule:

• 50% upon submission of the draft evaluation report;
• 50% upon submission of the final evaluation report.

Competencies

Corporate Competencies:

• Demonstrates integrity by modeling the UN’s values and ethical standards (human rights, peace, understanding between peoples; and nations, tolerance, integrity, respect, results orientation (UNDP core ethics) impartiality;
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

 Functional Competencies:

• Knowledge and experience with programming development, monitoring and evaluation;
• Consistently approaches work with energy and a positive, constructive attitude;
• Demonstrates openness to change, flexibility, and ability to manage complexities;
• Ability to work under pressure and with multi-disciplinary and multicultural teams and possess excellent inter-personal skills;
• Demonstrates strong written and oral communication skills;
• Remains calm, in control and good humored even under pressure;
• Proven networking, team-building, organizational and communication skills;
• Ability to establish priorities for self and others, and to work independently.

Required Skills and Experience

Education:

• A master’s degree in disaster management, climate change, geography, sustainable development or relevant discipline; Ph.D. an asset

Experience:

• Minimum 10 years of technical expertise in the field of sustainable development strategy/policy, disaster management, climate change, or relevant areas;
• Experience in conducting evaluations at programme and/or outcome levels in related fields with international organizations;
• Experience in implementing a range of qualitative and quantitative data collection techniques and methods in programme evaluations;
• Good knowledge of UNDP’s mandate and the political, cultural and economic contexts of the region.

Language:

• Fluency in English.

Evaluation of the Candidates:

 Individual consultants will be evaluated based on the following methodology.

 Cumulative analysis: The candidates will be evaluated through Cumulative Analysis method. The award of the contract will be made to the individual consultant whose offer has been evaluated and determined as:

• Responsive/compliant/acceptable, and
• Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

 Only candidates obtaining a minimum of 49 points out of 70 in the technical evaluation would be considered for Financial Evaluation.

 Technical Evaluation Criteria (Total 70 marks):

• Solid technical expertise in the field of sustainable development strategy/policy, disaster management, climate change or relevant areas (30 marks);
• Previous experience in conducting evaluations at programme and/or outcome levels in related fields with international organizations (20 marks);
• Technically sound approach in the proposed methodology (20 marks).

Financial Evaluation (Total 30 marks):

All technical qualified proposals will be scored out 30 based on the formula provided below. The maximum points (30) will be assigned to the lowest financial proposal. All other proposals received points according to the following formula:

p = y (µ/z)

Where:

p = points for the financial proposal being evaluated

y = maximum number of points for the financial proposal

µ = price of the lowest priced proposal

z = price of the proposal being evaluated

Financial Proposal

Total lump sum: The financial proposal shall specify a total lump sum amount, and payment terms around specific and measurable (qualitative and quantitative) deliverables (i.e. whether payments fall in installments or upon completion of the entire contract). Payments are based upon output, i.e. upon delivery of the services specified in the TOR.  In order to assist the requesting unit in the comparison of financial proposals, the financial proposal will include a breakdown of this lump sum amount (including travel, per diems, and number of anticipated working days).

Travel

All envisaged travel costs must be included in the financial proposal. This includes all travel to join duty station/repatriation travel.  In general, UNDP should not accept travel costs exceeding those of an economy class ticket. Should the IC wish to travel on a higher class he/she should do so using their own resources. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and Individual Consultant, prior to travel and will be reimbursed.

Documents To Be Included Whne Submitting The Proposal:

Interested individual consultants must submit the following documents/information to demonstrate their qualifications in one single PDF/Word file:

• Personal CV that provides a detailed description of pervious relevant experiences and technical expertise pertaining to this particular assignment;
• A methodology on how they will approach and complete the assignment (no more than two pages).