Country Context Yemen is widely considered to be the worst humanitarian and development crisis in the world. The conflict, which escalated in March 2015, has caused major loss of life, internal displacement, and destroyed critical infrastructure, government fragmentation, poor public service delivery, weakened population and institutional resilience and food insecurity verging on famine. Major roads and bridges across the country have been destroyed, power lines have been severely damaged, and oil and gas production are totally disrupted. An estimated 24.1 million people - equivalent to more than 80 % of the population need humanitarian or protection assistance, including 14.3 million in acute need. The socio-economic situation deteriorated further due to economic effects of COVID-19 including the reduction of remittances, business closures and livelihood losses; lower oil revenues due to reduced global oil prices and conflict-related disruption in oil production and exports; and reduced foreign aid – only 40% of the 2020 Humanitarian Response Plan was funded by September 2020. Many aid agencies scaled down humanitarian operations due to funding shortfalls, access constraints posed by the conflict, COVID-19, bureaucratic impediments. As a result, pockets of the population are exposed to famine-like conditions for the first time in two years. More than 20 million people are food insecure1, Prior to the crisis, solid waste management (SWM) in Yemen was based on the involvement of public and private stakeholders. Public stakeholders included, at the local level: (i) local authorities, which provided waste collection services; and (ii) city cleaning and improvement funds (CCIFs), which financed SWM service provision through local taxation and central government transfers, owned the equipment, and were responsible for its maintenance. Solid waste generation in Yemen was estimated to be 0.55–0.65 kilograms (kg) per capita in urban areas with a projected yearly increase of 3 percent on a national level, driven by population growth and increased rural-urban internal migration flows. Solid waste management was already problematic before the crisis; most waste was disposed of in dumpsites, and there were no structured and properly managed sanitary landfills2. The conflict has had a significant impact on key supply, sanitation, and hygiene (WASH) facilities around the country. According to the World Bank’s Yemen Dynamic Needs Assessment (2020), the conflict affected the solid waste management cycle in several ways: (i) waste collection slowed, due to a lack of fuel and damaged or stolen equipment (for example, collection trucks and vehicles); (ii) populations started to use alternatives to disposal sites; (iii) road access to some neighbourhoods became blocked; (iv) an influx of internally displaced persons (IDPs) placed additional pressure on the system; (v) unexploded ordnances (UXOs) and explosive remnants of war (ERWs) are present throughout the country, with reports of ERWs being found in waste accumulations in several cities; (vi) a reduction in the tax revenues of local authorities and CCIFs led to a halt in salary payments to public sector workers, including those in waste management; and (vii) the withdrawal of many international donors halted the implementation of the National Strategy for Solid Waste Management. It should also be noted that due to import restrictions and decreased food availability, waste generation rates have in general decreased. To restore Yemen’s solid waste management system back to its pre-crisis condition, investments in (i) waste removal; (ii) waste collection, transport, and disposal; (iii) waste transfer stations; and (iv) disposal facilities are needed. Waste removal supports public health by restoring an environmentally and hygienically acceptable situation3. The solid waste plant in Aden is located at Beer Al-Nea’ma on the west side of the city. The city’s population has increased from 900,000 in 2015 to over 1.6 million. Almost 50% of the solid waste collection trucks and bulldozers were stolen, and the cleaning fund was not increased to commensurate the ballooning solid waste generation. Aden water supply and drainage systems sustained serious damages; pumps were stolen or destroyed, and pipes were badly damaged. New human settlements were built over the main water drainage networks causing serious damage to the grid. The drainage system of the city requires urgent repair to maintain hygiene and avert the spread of diseases like cholera. The city of Mukalla’s WASH infrastructure was destroyed by a combination of natural disaster - typhoons and hurricanes and conflict. The growing population has also exerted pressure on the constrained facilities. The city has thus been unable to effectively deliver WASH services resulting in stagnant cesspools of water, the accumulation of waste and outbreak of diseases like dengue fever, typhoid, ophthalmia, malaria and cholera. Lack of service delivery is turning into a conflict driver in the relatively peaceful cities. Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla project was designed to enhance the capacities of key essential service SWM and WASH institutions in Aden and Mukalla to effectively deliver services to the population. Improved and sustained SWM and WASH will reduce the spread of epidemics, improve human security and provide peace dividends to Yemenis in and around the two targeted cities. The project contributes to the Humanitarian-Development Nexus, strengthening the resilience of two key institutions that are in a unique position to prevent health crises and improve general safety and human security. Fundamental improvements in performance and institutional strengthening assist in creating an environment for commercial activities. The project contributes to UNDP Country Programme Document Outcome 1: Yemenis contribute to and benefit from inclusive, accountable and gender responsive governance, at local and central levels and Sustainable Development Goal 5: Water and Sanitation. The project is funded by the Government of Japan has a budget of US$8.216M covers the period from 1 December 2018 to 31 May 2021. The project has two outputs; 1) Capacities of the Cleaning Fund and Public Corporation for Water and Sewerage (PCWS) are improved in Aden and Mukalla to address and adapt workload in a resilient manner; and 2) Critical infrastructure and Rehabilitation of pumping stations including main sewage network. The project was implemented through public corporations for water supply and sewerage (PCWS) and cleaning fund (CF) and national partners - Nahda Makers Organization (NMO) and Public works project (PWP). Purpose of the Evaluation Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla project ends on 31 May 2021. This final evaluation is commissioned to assess the project’s progress towards a) improved delivery of SWM and WASH services to the population in Aden and Mukalla, and b) reduce the spread of epidemics, improve human security and c) provide peace dividends to Yemenis in and around the two targeted cities. The final project evaluation serves as an important learning and accountability tool, providing the donor - Government of Japan, UNDP, key national stakeholders, and authorities in the targeted governorates and cities with an impartial assessment of the results generated, including gender equality measures and women’s empowerment. The evaluation will assess the project’s relevance, effectiveness, efficiency, impact, and sustainability, identify and document lessons learned, and provide recommendations to inform future project phases. The findings and recommendations of the evaluation will guide the key stakeholder, relevant Yemen institutions and authorities, project donor, UNDP, UN agencies, civil society organisations in implementation of related projects. Objectives Specific project evaluation objectives are to: 1. Assess the relevance and strategic positioning of Crisis Support for Solid Waste and Water Supply and Sewage institutions in Aden and Mukalla and whether the initial assumptions are still relevant. 2. Assess a) the progress made towards project results and whether there were any unintended results; b) what can be captured in terms of lessons learned for future SWM and WASH projects. 3. Assess whether the project management arrangements, approaches and strategies, including monitoring strategies and risk management approaches, were well-conceived and efficient in delivering the project. 4. Assess the overall contribution of the projects towards humanitarian-peace-development nexus and whether there are indications of sustaining the project’s results after the end of the project. 5. Analyse the extent to which the project enhanced application of a rights-based approaches, gender equality and women’s empowerment, social and environmental standards, and participation of other socially vulnerable groups such as children and the disabled. Scope The Project Evaluation will cover the period 1 December 2018 to 31 May 2021 covering the project locations – Aden and Mukalla. The evaluation will cover programme conceptualisation, design, implementation, monitoring, reporting and evaluation of results. The evaluation will engage all project stakeholders - benefitting communities/institutions, Aden and Mukalla city authorities, Government of Japan, UNDP, UN agencies and CSOs. The evaluation will assess progress made on key indicators agree with the donor. In addition to assessing the relevance, effectiveness, efficiency of the project, the evaluation will a) explore the key factors that have contributed to the achieving or not achieving of the intended results; and b) determine the extent to which the project contributed to improving SWM and WASH service delivery; addressing crosscutting issues of gender equality and women’s empowerment and human rights; and forging partnership at different levels, including with government, donors, UN agencies, and communities; c) assess potential sustainability of the project for continued realisation of results; and d) draw lessons learned and best practices and make recommendations for future SWM and WASH projects. Review Questions Referencing and adopting from Organization for Economic Cooperation and Development (OECD) Development Committee (DAC)4 evaluation criteria, the evaluation will answer the following questions: Relevance 1. Was the project relevant in addressing SWM and WASH service challenges in Aden and Mukalla? 2. To what extent was the project in line with the national development priorities, priorities of the cities of Aden and Mukalla, the country programme’s outputs and outcomes and the SDGs? 3. Was the project relevant to the needs and priorities of the target groups/beneficiaries including women and men? Were they consulted during design and implementation of the project? 4. Did the project address gender issues and help women overcome challenges or limitations? Effectiveness/Impact 5. To what extent has the Facility contributed to the country programme outcomes and outputs, the SDGs, the UNDP Strategic Plan and national peacebuilding priorities? 6. To what extent did the project achieve its intended results? 7. To what extent has the project contributed toward towards a) improved delivery of SWM and WASH services to the population in Aden and Mukalla, and b) reduce the spread of epidemics, improve human security and c) provide peace dividends to Yemenis in and around the two targeted cities. 8. To what extent did the project substantively mainstream a gender and support gender-responsive peacebuilding? 9. What factors have contributed to achieving or not achieving intended project outputs and outcomes? 10. To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights? 11. To what extent did COVID-19 impact positively and negatively to the project implementation? Efficiency 12. To what extent was the project management structure as outlined in the project document efficient in generating the expected results? 13. To what extent have the project implementation strategy and execution been efficient and cost-effective? 14. To what extent has there been an economical use of financial and human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes? 15. To what extent have the M&E systems utilized by UNDP enabled effective and efficient project management? 16. To what extent gender equality results are achieved at reasonable cost? Sustainability 17. To what extent will financial and economic resources be available to sustain the benefits achieved by the project? 18. Are there any social or political risks that may jeopardize sustainability of project outputs and the project’s contributions to country programme outputs and outcomes? 19. To what extent have relevant Ministries or national offices integrated project outcomes into ongoing policies and practices? 20. To what extent are lessons learned being documented by the project team on a continual basis and shared with appropriate parties who could learn from the project? 21. To what extent the interventions have well-designed and well-planned exit strategies? 22. Did sustainability differ for female and male beneficiaries? 23. Was the project financially and/or programmatically catalytic? Risk tolerance and innovation 24. If the project was characterized as “high risk”, were risks adequately monitoring and mitigated? 25. Was conflict sensitivity mainstreamed and included as an approach throughout project implementation? 26. How novel or innovative was the project approach? Can lessons be drawn to inform similar approaches elsewhere? Human rights 27. To what extent have poor, indigenous and physically challenged, women and other disadvantaged and marginalized groups included in helping design and prioritize the work of the project in the spirit of broad societal inclusion. To what extent have they benefited from the work of the project? 28. To what extent are the planned project interventions relevant to the overall strategy of inclusivity and empowerment? Gender equality and empowerment 29. To what extent have gender equality and the empowerment of women been addressed in the design, implementation, and monitoring of the project? 30. To what extend the commitment made to Gender Equality and Women Empowerment (GEWE) provisions of the project were realized in practice? 31. To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects? Guiding evaluation questions will be further refined by the evaluation team and agreed with UNDP evaluation stakeholders. Methodology The evaluation will be carried out in accordance with UNDP evaluation guidelines and policies, United Nations Group Evaluation Norms and Ethical Standards; OECD/DAC evaluation principles and guidelines and DAC Evaluation Quality Standards, with specific reference to the OECD DAC guidance on evaluation of peacebuilding initiatives. As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. Travel to and in the country is constrained by a combination of COVID-19 and the ongoing conflict. If it is not possible to travel to or within the country for the evaluation then the evaluation team should develop a methodology that takes this into account the conduct of evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, survey and evaluation questionnaires. This should be detailed in the Inception Report and agreed with the Evaluation Manager. If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the evaluation report. If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom, etc.). International consultant can work remotely with national evaluator support in the field if it is safe for them to operate and travel. No stakeholders, consultants or UNDP- should be put in harm’s way and safety is the key priority. A short validation mission may be considered if it is confirmed to be safe for staff, consultants, stakeholders and if such mission is possible within the evaluation schedule. Equally, qualified and independent national consultants can be hired to undertake the evaluation and interviews in country as long as it is safe to do so. It is expected that the evaluation will employ a combination of both qualitative and quantitative evaluation methods. The evaluation team should propose their own methodology, which may include: 1. Document review of all relevant documentation. This would include a review of inter alia; project document (contribution agreement); theory of change and results framework; programme and project quality assurance reports; annual workplans; consolidated midyear and annual reports; results-oriented monitoring report; highlights of project board meetings; and technical/financial monitoring reports. 2. Semi-structured interviews with key female and male stakeholders. This would include Aden and Mukalla city authorities, community leaders, project implementing agencies, representatives of key civil society organizations, UNCT members. ? Development of evaluation questions tailored to the different needs and participation of various stakeholders. ? All interviews will be undertaken in full confidence and anonymity. Prior to engaging in interviews or focus group discussions, the evaluation team must obtain written informed consent from all stakeholders, but especially those from vulnerable categories. The final evaluation report should not assign specific comments to individuals but indicate patterns according to categories of respondents. 3. Field visits and on-site validation of key tangible outputs and interventions. The evaluation team is expected to follow a participatory and inclusive consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct male and female beneficiaries. 4. Other methods such as outcome mapping, observational visits, group discussions, etc. 5. Data review and analysis of monitoring and other data sources and methods. 6. All analysis must be based on observed facts, evidence, and data. Findings should be specific, concise and supported by quantitative and/or qualitative information that is reliable, valid and generalizable. The broad range of data provides strong opportunities for triangulation. This process is essential to ensure a comprehensive and coherent understanding of the data sets, which will be generated by the evaluation. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators. Evaluation Ethics Evaluations in the UN are conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation.’ The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not or other uses with the express authorization of UNDP and partners. The Consultants are required to read the guidelines and ensure a strict adherence, including establishing protocols to safeguard confidentiality of information obtained during the evaluation. The Consultants upon signing the contract will also sign this guideline which may be made available as an attachment to the evaluation report.
|
Qualifications and competencies ? Degree level qualification in Civil engineering with a relevant masters degree in relevant disciplines (Governance, WASH, international development, social sciences, or related fields). ? At least 5 years of experience working on issues related to crisis response or social transformation projects in conflict environments. ? At least 5 years of experience in designing and leading program evaluation in a peacebuilding context, including with programming in relation to gender equality, women’s economic and political empowerment and peacebuilding and reconciliation. Implementation Arrangements The UNDP Yemen Country Office will select the consultants through an open process in consultation with the partners. UNDP will be responsible for the management of the consultant and will in this regard designate an evaluation manager and focal point. Project staff will assist in facilitating the process (e.g., providing relevant documentation, arranging visits/interviews with key informants, etc.). The evaluation manager will convene an evaluation reference group comprising of technical experts from UNDP the implementing partners. This reference group will review the inception report and the draft review report to provide detailed comments related to the quality of methodology, evidence collected, analysis and reporting. The reference group will also advise on the conformity of processes to the UNDP and UNEG standards. The consultants will take responsibility, with assistance from the project team, for setting up meetings and conducting the review, subject to advanced approval of the methodology submitted in the inception report. Project staff will not participate in meetings between consultants and evaluations participants. The consultants will report directly to the designated evaluation manager and focal point and work closely with the project team. The consultants of work full time and may be required to travel to the targeted areas for the purpose the evaluation. If it is not possible for the International Consultant to travel to Yemen due to security or COVID-19 restrictions, he/she should develop a methodology that takes this into account the conduct of evaluation virtually and remotely. This should be detailed in the Inception Report and agreed with the Evaluation Reference Group and the Evaluation Manager. Support during the implementation of remote/ virtual meetings will be provided by evaluation manager and focal point. An updated stakeholder list with contact details (phone and email) will need to be provided by the Country office to the consultants. UNDP with support of relevant stakeholders will develop a management response to the evaluation within 2 weeks of report finalization. Office space and limited administrative and logistical support will be provided. The consultants will use their own laptop and cell phone. Timeframe for Evaluation Process The project evaluation will be carried out over a period of 35 working days broken down as follows: Deliverable | Time allocated | % of payment | Documents to be submitted | Approving Officer | Evaluation inception report | | | | | A detailed inception (10-15 pages) report describing initial findings based on the comprehensive documentation review, the evaluation methodology, detailed work plan, the outline of the final report in addition to the inception report. Presentation and approval of IR by the ERG | 6 days | 20% | A comprehensive Inception Report | Team Leader, MSU | Data collection by applying all tools agreed in the inception report | | | | | Debriefing - powerpoint presentation | 21 days | 40% | A draft evaluation report | Team Leader, MSU | Draft evaluation report (max 40 pages) to be prepared based on collected data and information | Final evaluation report | | | | | Incorporation of comments and feedback on draft evaluation report provided by UNDP and other stakeholders | 8 days | 40% | Final evaluation report | Team Leader, MSU | Preparation of final evaluation report | Validation of the final draft, incorporation of validation comments and preparation and submission of final report | Power point for stakeholders | Final evaluation report along with audit trail | Total number of days | 35 days | 100% | | |
The International and National consultants will be jointly responsible for entire evaluation processes and submission of the above-mentioned deliverables. The 35 working days will be spread over a period of two months to provide for delays and the need for additional time that may be required for implementing evaluations virtually recognising possible delays in accessing stakeholder groups due to COVID-19. The consultants will inform the evaluation manager is additional time is needed to complete the evaluation. Technical proposals (total score: 70 points) Criteria | Max score | Weight | General adherence to the Term of Reference (ToR) | 5 | 7% | Proposed methodology, approach, and workplan (relevance, logic, rigor, practicality, creativity, realism of work plan etc). - Clarity and relevance of the proposed methodology, to the local context and to achieve the deliverables of the ToR.
- Realistic and complete work plan which reflects clear and comprehensive understanding of the scope of work in the ToR.
- Clarity about how gender considerations will be factored into the evaluation.
- Clarity on the quality assurance process that will be in place for this assignment
| 35 | 50% | Quality of plan to ensure ethics of conducting evaluation with human subjects (methodological component that will be accorded special attention given the project engagement of women, juvenile children, and other targeted groups). | 10 | 14% | Technical capacity of the applicant: qualifications, competencies, experience and skills as per the ToR (also assessed against sample of evaluation work done) | 20 | 29% | Total | 70 | 100% |
b) Financial Proposal (total score: 30 points) The financial proposal will specify a total lump sum amount and payment terms shall be in line with those that are mentioned in the deliverable table. Financial proposal will be assessed based on the completeness, clarity and appropriateness. The maximum number of points shall be allotted to the lowest Financial Proposal that is opened /evaluated and compared among those technical qualified candidates who have obtained a minimum 70 points in the technical evaluation. Other Financial Proposals will receive points in inverse proportion to the lowest price applying the formula: Marks Obtained = Lowest Priced Offer (Amount) / Offer being considered (Amount) X 30 (Full Marks) Payment Fee payments will be made upon acceptance and approval by UNDP planned deliverables, based on the following payment schedule: Milestone for payment | Percentage | Inception report | 17% | Draft Evaluation Report & presentation of findings | 60% | Final Evaluation Report, audit trail and presentation of findings to stakeholders | 23% |
In line with the UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactorily completed due to the impact of COVID19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her control. Documents to be provided by UNDP to successful candidates - Inception report
- Evaluation report
- Audit trail
- UNEG Code of Conduct for Evaluation in the UN system
- Integrating Gender Equality and Human Rights in Evaluation - UN-SWAP Guidance, Analysis and Good Practices
- UNDP Evaluation Guidelines
- Evaluation Quality Assessment
- UNEG Quality Checklist for Evaluation Reports
|