Background

The humanitarian crisis in Yemen has reached unprecedented levels of need with three-quarters of the population requiring some form of basic assistance to survive. The conflict has destroyed regular livelihoods of many Yemeni people and their ability to purchase adequate food required for their family members. It has also caused major disruptions in the food supply chain and increases in the price of imported foods, which Yemen is heavily dependent on, resulting in substantial increases in food prices. Almost two-thirds of the population are food insecure, 10 million people are at risk of famine, and two million children require treatment for acute malnutrition. Moreover, most of the population also lacks access to basic services such as education, health services, water and sanitation.
To respond to the effects of the conflict, a three-year European Union (EU) funded Social Protection for Community Resilience in Yemen (SPCRP) project was launched on September 2017 with a budget of EURO 25M. The project, implemented in partnership with the Social Fund for Development (SFD) aimed to:
· strengthen the resilience capacity of poor and vulnerable communities and households through improved access to public services and social safety nets, with particular attention to the inclusion of women, youth people, and marginalized minorities.
· mitigate the impact of the current crisis on local households and communities and assist their recovery from the bottom-up using local systems, strengthening capacities and institutions to progressively resume and scale-up service delivery, especially focusing on health services.
The project focused on achieving specific results in (1) increasing short-term income earning opportunities through labor intensive cash for work and asset upgrade and replacement for enhanced livelihoods and economic self-reliance of vulnerable Yemenis; (2) improving access to health care services (including psychosocial support) and related infrastructure through community based projects and provision of renewable energy (solar equipment); (3) restoring service delivery by strengthening the capacities of local authorities for planning, coordination and monitoring.
The project is implementing small, fast-disbursing interventions that serve as a rapid response, providing households and communities affected by the conflict with income support (as wages) to purchase basic necessities. These short-term interventions are also delivering benefits to the wider community by creating community assets, small infrastructure, and improved access to basic service delivery, as well as restoring livelihoods. The project adopts a community-based approach, which brings communities together around common humanitarian and development initiatives and hence promotes social cohesion and the protection of human capital.

As of December 2020, the total number of reported COVID-19 cases was 2,087 with 607 deaths and 1,385 recoveries in Yemen. Factors that have had a negative impact on the COVID-19 response in Yemen include a lack of adaptive behavior by the population to reduce transmission, severe funding shortages for health workers and personal protective equipment (PPE) and long delays in importing COVID-19 response supplies. The UN agencies along with other international and national partners continued working towards increasing surveillance; deploying dedicated COVID-19 staff within agencies; tracking the impact of the virus on routine priority health programmes; refining messaging to encourage behavioral change; and boosting intensive care unit (ICU) capacity. Partners have begun to prepare for a second wave of COVID-19, including engaging with excluded and vulnerable groups to keep them safe and supporting essential health facilities to ensure they function throughout winter. The fuel and funding crisis risked exacerbating the COVID-19 and broader humanitarian response as well.

Although, Project implementation gained momentum in 2019; yet, with COVID-19 outbreak in Yemen some activities were delayed and slow implementation due to restrictions of movements including those requiring crowds of people and training activities. However, project implementations were continued with taking precautionary measures to minimize the risk of infection during implementation. Moreover, immediate actions were taken into consideration to response to the COVID-19 included raising awareness and support communities create protective gear such as masks. The COVID-19 response initiatives provided a 12 days training for 239 women making face masks and personal protection equipment or suits for health workers. The training was based on the ministry of health guidelines and as a result, 18,000 cotton face masks were produced that are from cotton and can be used more than once after clearing or /washing them and 239 personal protective suits for health workers. Further, 192 Village Cooperation Councils (VCCs) implemented self-help activities as a contribution to the coronavirus awareness campaigns and distributed leaflets and hygiene and cleaning materials to people in their villages. The VCCs were coordinated with the district health workers.

Following delays in implementing project activities due to a constrained operating environment and outbreak of COVID-19 in 2020, UNDP requested a nine-month no-cost extension of the project to 31 March 2021.

2. Evaluation Purpose, Objectives and Scope

2.1 Purpose

With an anticipated SPCRP end date of 31 March 2021, this evaluation is being conducted to assess the project’s contribution towards strengthening the resilience capacity of poor and vulnerable communities and households, mitigating the impact of the current crisis on local households and communities and assisting institutions to progressively resume and scale-up service delivery, especially focusing on health services.

UNDP commissions a final project evaluation to serve as an important learning and accountability tool, providing the European Union, UNDP, national stakeholders, and partners with an impartial assessment of the results achieved by the project, including on gender equality and women empowerment. The evaluation also assesses the internal and external factors affecting the project’s outcomes. The evaluation will assess the project’s relevance, effectiveness, efficiency and sustainability; identify and document lessons learned; and provide recommendations to inform key stakeholders; relevant Yemeni ministries and institutions, Facility donors, UNDP, UN agencies, civil society organizations on any adjustment/redirection that may be necessary for future social safety net support in emergency contexts.

2.2 Evaluation Objectives:

The specific objectives of the evaluation are to:

1. Assess the relevance and strategic positioning of the project in strengthening the resilience capacity of poor and vulnerable communities and households, mitigating the impact of the current crisis on local households and communities and assisting institutions to progressively resume and scale-up service delivery, especially focusing on health services.

2. Assess a) the progress made towards project results and whether there were any unintended results and b) what can be captured in terms of lessons learned for ongoing and future UNDP’s social protection, resilience building and institutional capacity enhancement initiatives in Yemen.

3. Assess whether the project management arrangements, approaches and strategies were well-conceived and efficient in delivering the project.

4. Analyse the extent to which the project enhanced the application of a rights-based approaches, gender equality and women’s empowerment, social and environmental standards and participation of other socially vulnerable groups such as children and the disabled.

2.3 Evaluation Scope:

The evaluation will focus on the SPCRP interventions implemented by UNDP and its national implementing partner in targeted 13 governorates of Yemen from the project’s inception in September 2017 to March 2021.

The evaluation will cover the SPCRP conceptualization, design, implementation, monitoring, reporting and evaluation of results and will engage all project stakeholders. The evaluation will assess the relevance, effectiveness, efficiency and sustainability of the project; explore the key factors that have contributed to the achievement or non-achievement of planned results including the impact of COVID-19 pandemic; and determine the extent to which the project contributed to improving the resilience of vulnerable Yemenis and responded to COVID-19 to the targeted beneficiaries and communities; addressing crosscutting issues of gender equality and women’s empowerment and human rights; and forging partnership at different levels, including with government, donors, UN agencies, and communities.

3. Evaluation Criteria and Key Questions:

The evaluation will answer the following questions structured around the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) criteria of relevance, efficiency, effectiveness, and sustainability. Furthermore, the evaluation will also answer specific questions related to human rights, gender equality, and impact of COVID-19 and limitations to cover evaluation questions.

Relevance

1. To what extent was the project in line with the national development priorities, the country programme’s outputs and outcomes, the UNDP Strategic Plan and the SDGs?

2. To what extent does the project contribute to the theory of change for the relevant country programme outcome?

3. To what extent were lessons learned from other relevant projects considered in the project’s design?

4. To what extent were perspectives of those who could affect the outcomes, and those who could contribute information or other resources to the attainment of stated results, taken into account during the project design processes?

5. To what extent does the project contribute to gender equality, the empowerment of women and the human rights-based approach?

6. To what extent has the project been appropriately responsive to political, legal, economic, institutional, etc., changes in the country?

Effectiveness

7. To what extent did the project contribute to the country programme outcomes and outputs, the SDGs, the UNDP Strategic Plan and national development priorities?

8. To what extent were the project outputs achieved?

9. What factors have contributed to achieving or not achieving intended country programme outputs and outcomes?

10. To what extent has the project been appropriately responsive to the needs of the national constituents and changing partner priorities?

11. In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?

12. In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?

13. What, if any, alternative strategies would have been more effective in achieving the project’s objectives?

14. Are the projects objectives and outputs clear, practical and feasible within its frame?

15. To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights?

Efficiency

16. To what extent was the project management structure as outlined in the project document efficient in generating the expected results?

17. To what extent have the UNDP project implementation strategy and execution been efficient and cost-effective?

18. To what extent has there been an economical use of financial and human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?

19. To what extent have resources been used efficiently? Have activities supporting the strategy been cost-effective?

20. To what extent have project funds and activities been delivered in a timely manner?

21. To what extent do the M&E systems utilized by UNDP ensure effective and efficient project management?

Sustainability

22. Are there any financial risks that may jeopardize the sustainability of project outputs?

23. To what extent will financial and economic resources be available to sustain the benefits achieved by the project?

24. Are there any social or political risks that may jeopardize sustainability of project outputs and the project’s contributions to country programme outputs and outcomes?

25. To what extent are lessons learned being documented by the project team on a continual basis and shared with appropriate parties who could learn from the project?

26. To what extent do UNDP interventions have well-designed and well-planned exit strategies?

Human rights

27. To what extent have poor, indigenous and physically challenged, women and other disadvantaged and marginalized groups benefited from the work of UNDP in the country?

Gender equality

28. To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?

29. Is the gender marker data assigned to this project representative of reality?

30. To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

4. Methodology:

The evaluation will be carried out in accordance with UNDP evaluation guidelines and policies, United Nations Group Evaluation Norms and Ethical Standards; OECD/DAC evaluation principles and guidelines and DAC Evaluation Quality Standards.

The evaluation will employ a combination of both qualitative and quantitative evaluation methods including:

1. Document review of all relevant documentation. This includes a review of inter alia; Description of Action; theory of change and results framework; Contribution agreement, Letter of Agreement with SFD and proposal; project quality assurance reports; annual workplans; annual reports; EU results-oriented monitoring report; highlights of project board meetings; and technical/financial monitoring reports.

2. Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organizations, UNCT members and implementing partners:

? Development of evaluation questions around relevance, effectiveness, efficiency and sustainability will be designed for different stakeholders to be interviewed.

Key informant and focus group discussions with project, beneficiaries and stakeholders.

All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.

3. Field visits and on-site validation of key outputs and interventions. The evaluation team is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation manager, project team, implementing partners and direct beneficiaries.

4. Other methods such as outcome mapping, observational visits, group discussions, etc.

5. Data review and analysis of monitoring and other data sources and methods.

6. All analysis must be based on observed facts, evidence and data. Findings should be specific, concise and supported by quantitative and/or qualitative information that is reliable, valid and generalizable. The broad range of data provides strong opportunities for triangulation. This process is essential to ensure a comprehensive and coherent understanding of the data sets, which will be generated by the evaluation.

An outline on any adjusted evaluative approaches/ methodologies that may be needed to implement the evaluation effectively, including safety guidance, extended desk reviews, and virtual stakeholder meetings and interviews by evaluators should be detailed in the inception report. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should also be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders, and the evaluators.

5. Evaluation Products

The evaluation will be expected to deliver the following:

a) Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the subproject site visits.

b) Evaluation debriefings. Immediately following an evaluation, the evaluation team will conduct a preliminary debriefing of findings.

c) Draft evaluation report (max 40 pages). UNDP and stakeholders will review the draft evaluation report and provide an amalgamated set of comments to the evaluation team within 10 days, addressing the content required (as agreed in the inception report) and quality criteria as outlined in the UNDP evaluation guidelines.

d) Evaluation report audit trail. Comments and changes by the evaluation team in response to the draft report should be retained by the evaluators to show how they have addressed comments.

e) Final evaluation report.

f) Presentations to stakeholders and the evaluation reference group (ERG).

g) Evaluation brief and other knowledge products agreed in the inception report

The standard templates that need to be followed are provided in the Annexes section. It is expected that the evaluator will follow the UNDP evaluation guidelines and UNEG quality check list and ensure all the quality criteria are met in the evaluation report. The impact of COVID-19 on the production of deliverables may reduce the payment for consultants.

6. Implementation Arrangements:

The UNDP Yemen Country Office will select the consultants through a competitive process in line with UNDP’s rules and regulations. UNDP will be responsible for the management of the evaluation team and will in this regard designate an evaluation manager and focal point. Project staff will assist in facilitating the whole evaluation process - providing relevant documentation, constituting the evaluation reference group, arranging visits/interviews with key informants, reviewing evaluation deliverables stated above etc. An updated stakeholder list with contact details (phone and email) will need to be provided by the Project Management Unit (PMU) to the evaluation team.

The evaluation manager will convene an evaluation reference group comprising of technical experts from partners and UNDP to enhance the quality of the evaluation. The reference group will review the inception and the draft evaluation reports, providing detailed comments related to the quality of methodology, evidence collected, analysis and reporting. The reference group will also advise on the conformity of the evaluation process to UNDP and UNEG standards.

7. Evaluation Team and required competencies

The Evaluation Team will consist of experienced multi-disciplinary team of consultants (one international lead consultant and two national consultants, one for southern and one for northern region) with experience in designing and conducting evaluation for social safety net project/programme and humanitarian responses/actions in emergency contexts.

§ The International Consultant will be the Team Leader and take a lead role during all phases of the evaluation and coordinate the work of all other team members. He/she will ensure the quality of the evaluation process, outputs, methodology and timely delivery of all products. The Team Leader, in close collaboration with the other evaluation team members, leads the conceptualization and design the evaluation and plays a lead role in shaping the findings, conclusions, and recommendations of the report.

§ Two National Consultants (one for Southern and one for Northern Region) will be recruited to working closely with the international consultant (Team Leader), be responsible for the overall assistance to the Team Leader including collection and analysis of all relevant data from field and preparation of all consultations and meetings with selected different stakeholders. The National Consultants will contribute substantively to the work of the Team Leader, providing substantive advice and context in the drafting and finalizing the inception and final evaluation reports.

8. Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

9. Timeframe for the evaluation process

A tentative time frame for the evaluation is provided below. The evaluation is expected to be completed by 30 workdays within two months. This might be subject to change depending on the prevailing situation on ground at the time of the evaluation. However, the evaluation team should propose a timeline to submit the deliverables in their proposals with additional time for implementing evaluations virtually recognizing possible delays in accessing stakeholder groups due to COVID-19. Necessary and adequate time (at least two weeks) should be allocated for review and quality assurance processes of the deliverables by the UNDP Team and Partners.

Duties and Responsibilities

Key areas of national consultants to assist Evaluation Team Leader to accomplish the evaluation:

10. Method of Payment:

Payment is contingent on approval by the Project Manager upon satisfactory dispensing of the milestones and it will be paid in the instalments as stated in the deliverable table above.

11.Institutional Arrangements:

The national consultants will work under the direct supervision and guidance of the evaluation team leader (International Consultant) and take responsibility, with assistance from the project team, for setting up meetings, consultations and conducting field visits at project sites, subject to advance approval of the methodology submitted in the inception report. The national consultants will report directly to the evaluation team leader and evaluation manager, and work closely with the project team. Project staff will not participate in the meetings between consultants and evaluands. The consultants will work full time, based in UNDP Yemen and will be required to travel to the project sites as part of the evaluation (unless the situation dictates otherwise). Office space and limited administrative and logistical support will be provided. The consultants will use their own laptops and cell phone. UNDP will develop a management response to the evaluation within two weeks of report finalization.

Assessment and Weighting Criteria of the Proposals

Required mentioned documents to be included when submitting the Proposal: Interested individual consultants must submit the following documents/information to demonstrate their qualifications and interest: (i) Letter of Confirmation of Interest and Availability using the template provided by UNDP; (ii) Personal detailed CV including past experience in similar assignment and at least 3 references; (iii) UN P11 Form (“CV Form”); (iv) A brief Methodology on how the candidate will approach and conduct the work.

The proposals will be weighed according to the technical (70%) and financial considerations (30%). Submitted proposals will be assessed using Cumulative Analysis Method. Technical proposals should attain a minimum of 70 points to qualify and to be considered. Financial proposals will be opened only for those application that attained 70 or above. Below are the criteria and points for technical and financial proposals.

a) Technical proposals ( TOTAL score: 70 points)

b) Financial Proposal (total score: 30 points)

Price proposal shall be fixed lump-sum, all-inclusive price. Financial proposal will be assessed based on the completeness, clarity and appropriateness. The maximum number of points shall be allotted to the lowest Financial Proposal that is opened /evaluated and compared among those technical qualified candidates who have attained a minimum 70 points in the technical evaluation. Other Financial Proposals will receive points in inverse proportion to the lowest price applying the formula:

Marks Obtained = Lowest Priced Offer (Amount) / Offer being considered (Amount) X 30 (Full Marks)

15.Document to be provided by PMU

The list of available documents to be provided to the evaluation team immediately upon signing the contract. The list of documents are Approved Project Document; Letter of Agreement (LOA) between UNDP and National Implementing Partner; Approved AWPs; Final Project Progress Reports; Third Party Monitoring Reports; PMU Field Visit Reports; Lessons Learned Study/Assessment Report; and any other required documents to be provided as per request of evaluation team.

Competencies

Key Competencies expected of the Consultants for the successful delivery of the milestones:

a) Extensive national experience in assessing social safety net programmes, humanitarian emergency responses and support.

b) Expertise in applying results and human rights-based approaches for assessing social safety net, humanitarian response actions/programmes.

c) Strong familiarity with the international literature and issues related to humanitarian contexts. Solid knowledge and skill on UNDP work in social safety net, emergencies, and work experience with UNDP is an advantage.

d) Familiarity with the socio-cultural context of Middle East and the cultural, political and religious sensitivities relevant to the Yemen crisis.

e) Excellent writing and communication skills in English and Arabic, with sufficient experience in applying all required tools and methods for conducting project evaluation in crisis settings including participatory appraisal techniques in data collection, sensitive to gender issues etc.

Required Skills and Experience

The National Consultants should have the following qualifications:

a) The candidates should have at least a master’s degree in Economics/Development Study/Social Sciences/ Statistics/or related field especially advanced academic certificate/diploma courses on International Humanitarian Action will be added value.

b) At least 5 years for national consultant relevant work experience and must have completed at least two high quality evaluations in the areas of social safety net programme, humanitarian response or actions, at least one of them being related to emergency crisis response support in any war/conflict country. Provision of sample work is required. Team management skills and experience of working as a member of evaluation team to be considered as one of key technical competencies for the national consultant.