Background

Country Context
Yemen is widely considered to be the worst humanitarian and development crisis in the world. The conflict, which escalated in March 2015, has caused major loss of life, internal displacement, and destroyed critical infrastructure, government fragmentation, poor public service delivery, weakened population and institutional resilience and food insecurity verging on famine. Major roads and bridges across the country have been destroyed, power lines have been severely damaged, and oil and gas production are totally disrupted. An estimated 24.1 million people - equivalent to more than 80 % of the population need humanitarian or protection assistance, including 14.3 million in acute need.
The socio-economic situation deteriorated further due to economic effects of COVID-19 including the reduction of remittances, business closures and livelihood losses; lower oil revenues due to reduced global oil prices and conflict-related disruption in oil production and exports; and reduced foreign aid – only 40% of the 2020 Humanitarian Response Plan was funded by September 2020. Many aid agencies scaled down humanitarian operations due to funding shortfalls, access constraints posed by the conflict, COVID-19, bureaucratic impediments. As a result, pockets of the population are exposed to famine-like conditions for the first time in two years. More than 20 million people are food insecure1,
Prior to the crisis, solid waste management (SWM) in Yemen was based on the involvement of public and private stakeholders. Public stakeholders included, at the local level: (i) local authorities, which provided waste collection services; and (ii) city cleaning and improvement funds (CCIFs), which financed SWM service provision through local taxation and central government transfers, owned the equipment, and were responsible for its maintenance. Solid waste generation in Yemen was estimated to be 0.55–0.65 kilograms (kg) per capita in urban areas with a projected yearly increase of 3 percent on a national level, driven by population growth and increased rural-urban internal migration flows. Solid waste management was already problematic before the crisis; most waste was disposed of in dumpsites, and there were no structured and properly managed sanitary landfills2.
The conflict has had a significant impact on key supply, sanitation, and hygiene (WASH) facilities around the country. According to the World Bank’s Yemen Dynamic Needs Assessment (2020), the conflict affected the solid waste management cycle in several ways: (i) waste collection slowed, due to a lack of fuel and damaged or stolen equipment (for example, collection trucks and vehicles); (ii) populations started to use alternatives to disposal sites; (iii) road access to some neighbourhoods became blocked; (iv) an influx of internally displaced persons (IDPs) placed additional pressure on the system; (v) unexploded ordnances (UXOs) and explosive remnants of war (ERWs) are present throughout the country, with reports of ERWs being found in waste accumulations in several cities; (vi) a reduction in the tax revenues of local authorities and CCIFs led to a halt in salary payments to public sector workers, including those in waste management; and (vii) the withdrawal of many international donors halted the implementation of the National Strategy
for Solid Waste Management. It should also be noted that due to import restrictions and decreased food availability,
waste generation rates have in general decreased.
To restore Yemen’s solid waste management system back to its pre-crisis condition, investments in (i) waste removal; (ii) waste collection, transport, and disposal; (iii) waste transfer stations; and (iv) disposal facilities are needed. Waste removal supports public health by restoring an environmentally and hygienically acceptable situation3. The solid waste plant in Aden is located at Beer Al-Nea’ma on the west side of the city. The city’s population has increased from 900,000 in 2015 to over 1.6 million. Almost 50% of the solid waste collection trucks and bulldozers were
stolen, and the cleaning fund was not increased to commensurate the ballooning solid waste generation. Aden water supply and drainage systems sustained serious damages; pumps were stolen or destroyed, and pipes were badly damaged. New human settlements were built over the main water drainage networks causing serious damage to the grid. The drainage system of the city requires urgent repair to maintain hygiene and avert the spread of diseases like cholera.
The city of Mukalla’s WASH infrastructure was destroyed by a combination of natural disaster - typhoons and hurricanes and conflict. The growing population has also exerted pressure on the constrained facilities. The city has thus been unable to effectively deliver WASH services resulting in stagnant cesspools of water, the accumulation of waste and outbreak of diseases like dengue fever, typhoid, ophthalmia, malaria and cholera. Lack of service delivery is turning into a conflict driver in the relatively peaceful cities. Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla project was designed to enhance the capacities of key essential service SWM and WASH institutions in Aden and Mukalla to effectively deliver services to the population. Improved and sustained SWM and WASH will reduce the spread of epidemics, improve human security and provide peace dividends to Yemenis in and around the two targeted cities. The project contributes to the Humanitarian-Development Nexus, strengthening the resilience of two key institutions that are in a unique position to prevent health crises and improve general safety and human security. Fundamental improvements in performance and institutional strengthening assist in creating an environment for commercial activities. The project contributes to UNDP Country Programme Document Outcome 1: Yemenis contribute to and
benefit from inclusive, accountable and gender responsive governance, at local and central levels and Sustainable Development Goal 5: Water and Sanitation. The project is funded by the Government of Japan has a budget of US$8.216M covers the period from 1 December 2018 to 31 May 2021. The project has two outputs; 1) Capacities of the Cleaning Fund and Public Corporation for Water and Sewerage (PCWS) are improved in Aden and Mukalla to address and adapt workload in a resilient manner; and 2) Critical infrastructure and Rehabilitation of pumping stations including main sewage network.
The project was implemented through public corporations for water supply and sewerage (PCWS) and cleaning fund (CF) and national partners - Nahda Makers Organization (NMO) and Public works project (PWP).

Purpose of the Evaluation

Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla project ends on 31 May 2021. This final evaluation is commissioned to assess the project’s progress towards a) improved delivery of SWM and WASH services to the population in Aden and Mukalla, and b) reduce the spread of epidemics, improve human security and c) provide peace dividends to Yemenis in and around the two targeted cities. The final project evaluation serves as an important learning and accountability tool, providing the donor - Government of Japan, UNDP, key national stakeholders, and authorities in the targeted governorates and cities with an impartial assessment of the results generated, including gender equality measures and women’s empowerment. The evaluation will assess the project’s relevance, effectiveness, efficiency, impact, and sustainability, identify and document lessons learned, and provide recommendations to inform future project phases. The findings and recommendations of the evaluation will guide the key stakeholder, relevant Yemen institutions and authorities, project donor, UNDP, UN
agencies, civil society organisations in implementation of related projects.

Objectives

Specific project evaluation objectives are to:
1. Assess the relevance and strategic positioning of Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla and whether the initial assumptions are still relevant.
2. Assess a) the progress made towards project results and whether there were any unintended results; b) what can be captured in terms of lessons learned for future SWM and WASH projects.
3. Assess whether the project management arrangements, approaches and strategies, including monitoring strategies and risk management approaches, were well-conceived and efficient in delivering the project.
4. Assess the overall contribution of the projects towards humanitarian-peace-development nexus and whether there are indications of sustaining the project’s results after the end of the project.
5. Analyse the extent to which the project enhanced application of a rights-based approaches, gender equality and women’s empowerment, social and environmental standards, and participation of other socially vulnerable groups such as children and the disabled.

Scope

The Project Evaluation will cover the period 1 December 2018 to 31 May 2021 covering the project locations – Aden and Mukalla. The evaluation will cover programme conceptualisation, design, implementation, monitoring, reporting and evaluation of results. The evaluation will engage all project stakeholders - benefitting communities/institutions, Aden and Mukalla city authorities, Government of Japan, UNDP, UN agencies and CSOs. The evaluation will assess progress made on key indicators agree with the donor. In addition to assessing the relevance, effectiveness, efficiency of the project, the evaluation will a) explore the key factors that have contributed to the achieving or not achieving of the intended results; and b) determine the extent to which the project contributed to improving SWM and WASH service delivery; addressing crosscutting issues of gender equality and women’s empowerment and human rights; and forging partnership at different levels, including with government, donors, UN agencies, and communities; c) assess potential sustainability of the project for continued realisation of results; and d)
draw lessons learned and best practices and make recommendations for future SWM and WASH projects.

Review Questions

Referencing and adopting from Organization for Economic Cooperation and Development (OECD) Development Committee (DAC)4 evaluation criteria, the evaluation will answer the following questions:
Relevance
1. Was the project relevant in addressing SWM and WASH service challenges in Aden and Mukalla?
2. To what extent was the project in line with the national development priorities, priorities of the cities of Aden and Mukalla, the country programme’s outputs and outcomes and the SDGs?
3. Was the project relevant to the needs and priorities of the target groups/beneficiaries including women and men? Were they consulted during design and implementation of the project?
4. Did the project address gender issues and help women overcome challenges or limitations?
Effectiveness/Impact
5. To what extent has the Facility contributed to the country programme outcomes and outputs, the SDGs, the UNDP Strategic Plan and national peacebuilding priorities?
6. To what extent did the project achieve its intended results?

7. To what extent has the project contributed toward towards a) improved delivery of SWM and WASH services to the population in Aden and Mukalla, and b) reduce the spread of epidemics, improve human security and c) provide peace dividends to Yemenis in and around the two targeted cities.
8. To what extent did the project substantively mainstream a gender and support gender-responsive peacebuilding?
9. What factors have contributed to achieving or not achieving intended project outputs and outcomes?
10. To what extent has the project contributed to gender equality, the empowerment of women and the realization of
human rights?
11. To what extent did COVID-19 impact positively and negatively to the project implementation?
Efficiency
12. To what extent was the project management structure as outlined in the project document efficient in generating
the expected results?
13. To what extent have the project implementation strategy and execution been efficient and cost-effective?
14. To what extent has there been an economical use of financial and human resources? Have resources (funds, human
resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
15. To what extent have the M&E systems utilized by UNDP enabled effective and efficient project management?
16. To what extent gender equality results are achieved at reasonable cost?
Sustainability
17. To what extent will financial and economic resources be available to sustain the benefits achieved by the project?
18. Are there any social or political risks that may jeopardize sustainability of project outputs and the project’s
contributions to country programme outputs and outcomes?
19. To what extent have relevant Ministries or national offices integrated project outcomes into ongoing policies and
practices?
20. To what extent are lessons learned being documented by the project team on a continual basis and shared with
appropriate parties who could learn from the project?
21. To what extent the interventions have well-designed and well-planned exit strategies?
22. Did sustainability differ for female and male beneficiaries?
23. Was the project financially and/or programmatically catalytic?
Risk tolerance and innovation
24. If the project was characterized as “high risk”, were risks adequately monitoring and mitigated?
25. Was conflict sensitivity mainstreamed and included as an approach throughout project implementation?
26. How novel or innovative was the project approach? Can lessons be drawn to inform similar approaches elsewhere?
Human rights
27. To what extent have poor, indigenous and physically challenged, women and other disadvantaged and marginalized
groups included in helping design and prioritize the work of the project in the spirit of broad societal inclusion. To
what extent have they benefited from the work of the project?
28. To what extent are the planned project interventions relevant to the overall strategy of inclusivity and
empowerment?
Gender equality and empowerment
29. To what extent have gender equality and the empowerment of women been addressed in the design,
implementation, and monitoring of the project?
30. To what extend the commitment made to Gender Equality and Women Empowerment (GEWE) provisions of the
project were realized in practice?
31. To what extent has the project promoted positive changes in gender equality and the empowerment of women?
Were there any unintended effects?
Guiding evaluation questions will be further refined by the evaluation team and agreed with UNDP evaluation
stakeholders.

Methodology

The evaluation will be carried out in accordance with UNDP evaluation guidelines and policies, United Nations Group Evaluation Norms and Ethical Standards; OECD/DAC evaluation principles and guidelines and DAC Evaluation Quality Standards, with specific reference to the OECD DAC guidance on evaluation of peacebuilding initiatives. As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. Travel to and in the country is constrained by a combination of
COVID-19 and the ongoing conflict. If it is not possible to travel to or within the country for the evaluation then the evaluation team should develop a methodology that takes this into account the conduct of evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, survey and evaluation questionnaires. This should be detailed in the Inception Report and agreed with the Evaluation Manager.
If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the evaluation report. If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom, etc.). International consultant can work remotely with national evaluator support in the field if it is safe for them to operate and travel. No stakeholders, consultants or UNDP- should be put in harm’s way and safety is the key priority. A short validation mission may be considered if it is confirmed to be safe for staff, consultants, stakeholders and if such
mission is possible within the evaluation schedule. Equally, qualified and independent national consultants can be hired to undertake the evaluation and interviews in country as long as it is safe to do so. It is expected that the evaluation will employ a combination of both qualitative and quantitative evaluation methods. The evaluation team should propose their own methodology, which may include:
1. Document review of all relevant documentation. This would include a review of inter alia; project document (contribution agreement); theory of change and results framework; programme and project quality assurance reports; annual workplans; consolidated midyear and annual reports; results-oriented monitoring report; highlights of project board meetings; and technical/financial monitoring reports.
2. Semi-structured interviews with key female and male stakeholders. This would include Aden and Mukalla city authorities, community leaders, project implementing agencies, representatives of key civil society organizations, UNCT members.
? Development of evaluation questions tailored to the different needs and participation of various stakeholders.
? All interviews will be undertaken in full confidence and anonymity. Prior to engaging in interviews or focus group discussions, the evaluation team must obtain written informed consent from all stakeholders, but especially those from vulnerable categories. The final evaluation report should not assign specific comments to individuals but indicate patterns according to categories of respondents.
3. Field visits and on-site validation of key tangible outputs and interventions. The evaluation team is expected to follow a participatory and inclusive consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct male and female beneficiaries.
4. Other methods such as outcome mapping, observational visits, group discussions, etc.
5. Data review and analysis of monitoring and other data sources and methods.
6. All analysis must be based on observed facts, evidence, and data. Findings should be specific, concise and supported by quantitative and/or qualitative information that is reliable, valid and generalizable. The broad range of data provides strong opportunities for triangulation. This process is essential to ensure a comprehensive and coherent understanding of the data sets, which will be generated by the evaluation. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators.

Evaluation Ethics

Evaluations in the UN are conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation.’ The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not
 or other uses with the express authorization of UNDP and partners. The Consultants are required to read the guidelines and ensure a strict adherence, including establishing protocols to safeguard confidentiality of information obtained during the evaluation. The Consultants upon signing the contract will also sign this guideline which may be made available as an attachment to the evaluation report.

Duties and Responsibilities

Review products/Deliverables

The consultants /evaluation team will be expected to deliver the following:
a) Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review. It should be produced and approved before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of the international consultant.
b) Evaluation debriefings. Immediately following an evaluation, UNDP will ask for a preliminary debriefing of findings.
c) Draft evaluation report (max 40 pages). GoJ, UNDP and stakeholders will review the draft evaluation report and provide an amalgamated set of comments to the evaluator within 10 days, addressing the content required (as agreed in the inception report) and quality criteria as outlined in the UNDP evaluation guidelines.
d) Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
e) Final evaluation report. The final report should address comments, questions and clarification. The final report should also contain a stand-alone executive summary of no more than five pages. Standard templates that need to be followed are provided in the Annexes section. It is expected that the evaluator will follow the UNDP evaluation guidelines and UNEG quality check list and ensure all the quality criteria are met in the evaluation report.
In line with UNDP’s financial regulations, when determined by the Country Office and/or the consultants that a deliverable or service cannot be satisfactory completed due to impact of COVID-19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultants invested time towards the deliverable but was unable to complete to circumstances beyond his/her/their control.

Required Qualifications

The project evaluation will be conducted by three independent consultants - an international consultant and two national consultant. The consultants must have extensive experience in strategic programming of development assistance in active conflict setting countries within the broader areas of SWM, WASH and post conflict/disaster reconstruction. The consultants must also have substantial knowledge and experience of gender and monitoring and evaluation of similar initiatives in volatile environments. UNDP seeks to recruit three individual consultants – and international and two national to conduct a joint independent final evaluation. As part of the two-person evaluation team, the International Consultant will oversee, predominantly remote capacities, the methodological approach, ensure the quality assurance and provide technical support to the National Consultants to lead and carry out the necessary fieldwork and complete set of deliverables. The evaluation will be a participatory, consultative multi-stakeholder process focused on assessing results and the process towards the peacebuilding impact of the project implemented.

The required qualifications and technical competencies are listed below:
National Consultant – Independent Evaluator
Responsibilities of the national consultant
The National Consultant will be responsible for performing the following tasks under the guidance of the International
Consultant:
? Review documents and provide substantive support to defining evaluation scope, methodology and work plan.
? Support to develop research design and questions.
? Contribute to the production of the inception report.
? Data collection as per the approved inception report and allocation of responsibilities as agreed with the team
leader.
? Data analysis and drafting of report, and support/co-present the findings.
? Draft related parts of the evaluation report as agreed on the division of labour with the International Consultant.
? Assist the International Consultant in finalizing the evaluation report through incorporating suggestions received

Competencies

Qualifications and competencies
? Degree level qualification in Civil engineering with a relevant masters degree in relevant disciplines (Governance,
WASH, international development, social sciences, or related fields).
? At least 5 years of experience working on issues related to crisis response or social transformation projects in
conflict environments.
? At least 5 years of experience in designing and leading program evaluation in a peacebuilding context, including
with programming in relation to gender equality, women’s economic and political empowerment and
peacebuilding and reconciliation.
 

Implementation Arrangements

The UNDP Yemen Country Office will select the consultants through an open process in consultation with the partners. UNDP will be responsible for the management of the consultant and will in this regard designate an evaluation manager and focal point. Project staff will assist in facilitating the process (e.g., providing relevant documentation, arranging visits/interviews with key informants, etc.). The evaluation manager will convene an evaluation reference group comprising of technical experts from UNDP the implementing partners. This reference group will review the inception report and the draft review report to provide detailed comments related to the quality of methodology, evidence collected, analysis and reporting. The reference group will also advise on the conformity of processes to the UNDP and UNEG standards. The consultants will take responsibility, with assistance from the project team, for setting up meetings and conducting the review, subject to advanced approval of the methodology submitted in the inception report. Project staff will not participate in meetings between consultants and evaluations participants.
The consultants will report directly to the designated evaluation manager and focal point and work closely with the project team. The consultants of work full time and may be required to travel to the targeted areas for the purpose the evaluation. If it is not possible for the International Consultant to travel to Yemen due to security or COVID-19 restrictions, he/she should develop a methodology that takes this into account the conduct of evaluation virtually and remotely. This should be detailed in the Inception Report and agreed with the Evaluation Reference Group and the Evaluation Manager. Support during the implementation of remote/ virtual meetings will be provided by evaluation manager and focal point. An updated stakeholder list with contact details (phone and email) will need to be provided by the Country office to the consultants. UNDP with support of relevant stakeholders will develop a management response to the evaluation within 2 weeks of report finalization. Office space and limited administrative and logistical support will be provided. The consultants will use their own laptop and cell phone.

Timeframe for Evaluation Process

The project evaluation will be carried out over a period of 35 working days broken down as follows:

The International and National consultants will be jointly responsible for entire evaluation processes and submission of the above-mentioned deliverables.

The 35 working days will be spread over a period of two months to provide for delays and the need for additional time that may be required for implementing evaluations virtually recognising possible delays in accessing stakeholder groups due to COVID-19. The consultants will inform the evaluation manager is additional time is needed to complete the evaluation.

Technical proposals (total score: 70 points)

b) Financial Proposal (total score: 30 points)

The financial proposal will specify a total lump sum amount and payment terms shall be in line with those that are mentioned in the deliverable table.

Financial proposal will be assessed based on the completeness, clarity and appropriateness. The maximum number of points shall be allotted to the lowest Financial Proposal that is opened /evaluated and compared among those technical qualified candidates who have obtained a minimum 70 points in the technical evaluation. Other Financial Proposals will receive points in inverse proportion to the lowest price applying the formula:

Marks Obtained = Lowest Priced Offer (Amount) / Offer being considered (Amount) X 30 (Full Marks)

Payment

Fee payments will be made upon acceptance and approval by UNDP planned deliverables, based on the following payment schedule: 

In line with the UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactorily completed due to the impact of COVID19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her control.

Documents to be provided by UNDP to successful candidates

1. Inception report
2. Evaluation report
3. Audit trail
4. UNEG Code of Conduct for Evaluation in the UN system
5. Integrating Gender Equality and Human Rights in Evaluation - UN-SWAP Guidance, Analysis and Good Practices
6. UNDP Evaluation Guidelines
7. Evaluation Quality Assessment
8. UNEG Quality Checklist for Evaluation Reports

Required Skills and Experience

- Proven knowledge and understanding of M&E methodologies, including qualitative and quantitative data analysis
skills and participatory data collection approaches.
- Proven knowledge on results-based management.
- Proven knowledge of the social-economic and political dynamics in Yemen and/or the setting in which the project
activities have been implemented.
- Gender experience is preferred.

Due to COVID-19 and restrictions in travelling, the evaluation team can operate remotely, such as international
consultant. Include a provision for experience in implementing evaluations remotely. Females are encouraged to apply.

Application Process

Interested qualified and experienced individual consultants must submit the following documents/information to
demonstrate their qualifications and interest:

(i) Letter of Confirmation of interest and availability using the template provided by UNDP; see attached forma.

(ii) Most Updated Personal detailed CV including past experience in similar assignment and at least 3 references;

(iii) UN P11 Form (“CV Form”);

(iii) A detailed methodology on how the candidate will approach and conduct the work and

(v) links to at least two samples of evaluation work done/authored within the past two years. Submitted proposals will be assessed using Cumulative Analysis Method.

The proposals will be weighed according to the technical proposal (carrying 70%) and financial proposal (carrying 30%). Technical proposals should obtain a minimum of 70 points to qualify and to be considered. Financial proposals will be opened only for those application that obtained 70 or above in the technical proposal. Below are the criteria and points for technical and financial proposals.