Historique

The United Nation’s FAO, IOM, UNDP, UNICEF, UNIDO, and WFP, in close cooperation with the Government of Armenia (GOA), are jointly implementing the following project which is funded by the United Nations Trust Fund for Human Security; “Enhancing Human Security and Building a Resilient Society in the Disadvantaged Communities of Armenia”

The goal of the project is to support Armenia’s efforts in achieving the 2030 Agenda for Sustainable Development by addressing the root causes of human insecurity for vulnerable people of Amasia, Alaverdi, Tumanyan and Berd communities in Shirak, Lori and Tavush regions of Armenia. The Project aims to:

1. Ensure early prevention and sustainability of interventions through identification of root causes of threats to human security and enhancing community resilience;
2. Strengthen social protection and inclusion to improve human security in targeted communities;
3. Address the economic and food insecurity in the target communities through strengthened livelihoods, creation of sustainable economic opportunities and capacity building;

The human security approach (HSA) taken within the project addresses, individual rights, good governance, access to safe and inclusive education and health care, ensuring that opportunities and choices are fulfilled at maximum potential, and are aimed at reducing poverty, achieving economic growth and community resilience, as well as promoting understanding of human security in terms of the risks and insecurities faced by individuals and groups at grass roots level. This is achieved by identifying the specific needs of populations under stress, human security highlights the complexity of the challenges and promotes integrated solutions that ensure greater coherence and stronger impact.

The comprehensive community profiles, formulated with the support of the Project[1], as well as progress reports, mid-term evaluation reviews and project results framework will be provided  serving  as a baseline for monitoring and evaluation of the project results.

The project is looking to conduct an independent final evaluation, which will include an overall assessment of the project, vis-à-vis the following criteria: i) relevance; ii) effectiveness; iii) efficiency; iv) impact (to the possible extent) and v) sustainability achieved through joint input from all the participating agencies and national implementing partners, as well as recommendations for future interventions to build upon the results of the project.

UNDP Armenia, as the lead agency in the implementation of this joint project, seeks the services of a specialised  consultant in the above-mentioned area to lead a team of two independent consultants (Lead Evaluator and Evaluation Support Assistant/Evaluator) to conduct the final evaluation of the “Enhancing Human Security and building resilient societies in disadvantaged communities of Armenia” Project (hereinafter -the Project).

The successful candidate is expected , should the COVID-19 permit, in Yerevan, Armenia for live interviews. If Covid-19 restrictions do not permit this the evaluation can be conducted online with prior approval from UNDP Armenia.

[1] Participatory vulnerability and capacity assessments and Science based multi-risk assessments were conducted in all communities, will be provided.

Basic project information is shown below in a table format:

The project has its own result framework with three objectives, corresponding outputs and activities, as well as an indicator framework.

Devoirs et responsabilités

The overall objective of the final evaluation assignment is to assess the relevance, effectiveness, efficiency and sustainability of interventions under the “Enhancing Human Security and Building a Resilient Society in the Disadvantaged Communities of Armenia” Project.

Given that this is a final evaluation, impact will be evaluated to the extent possible.

The geographical coverage of the evaluation includes Tumanyan, Alaverdi, Berd and Amasia consolidated communities together with their settlements. Considering the three-year duration of the project, as well as the granted six months no cost extension, the final evaluation will be conducted for the period of October 2018- 31 March 2022.

The evaluation will focus on direct and indirect beneficiaries of the project interventions and its methodology, including the Evaluation Design, detailed Evaluation Matrix, Data Collection Tools and the Time-frame of the Evaluation will be spelled out in the inception report.

The evaluation will be carried out in close cooperation with the representatives from the above-mentioned six UN Agencies based on the project document, its results framework and the joint work-plan, using a combination of qualitive and quantitative data collection methods. Findings of the final evaluation report will be shared with the Project Board, implementing partners and the donor –UNTFHS, as well as posted on UNDP Evaluation Resource Centre public website (https://erc.undp.org). The evaluation findings will be shared with the members of Project board, regional and community administrations, participating NGOs and donor to inform on the results of the Project, and the lessons learned. It will also inform on further joint work between the implementing agencies towards the implementation of UNSDCF and mainstreaming the Human Security Approach at national and community levels.

The following stakeholders will be consulted during the evaluation:

Key Stakeholders:

• Ministry of Territorial Administration and Infrastructure including the Migration Service
• Ministry of Labour and Social Affairs
• Ministry of Economy
• Ministry of Education, Science, Culture and Sport
• Heads of targeted Amasia, Alaverdi, Berd and Tumanyan consolidated communities
• Project Beneficiaries

Other stakeholders:

• Development partners (UNTFHS)
• Relevant private sector and civil society organisations representatives

Evaluation Criteria and Key guiding questions:

All Project related documents and materials will be thoroughly reviewed in the Inception phase by the Evaluator(s) to finalize the evaluation design with a clear Evaluation Matrix, a clear logic and workplan of the evaluation, which shall be agreed by the evaluation commissioning unit in consultation with the implementing partners and relevant key stakeholders as deemed necessary.

Five core OECD-DAC evaluation criteria, namely the relevance, effectiveness, efficiency, sustainability and impact will be analysed. Guiding evaluation questions will include, but are not limited to, the following:

Relevance

• Are the project activities/components relevant to the actual/defined needs of the beneficiaries? Were the objectives clear and feasible? How do the main components of the project contribute to the planned objectives and are logically interlinked?
• Is the project in line with the current priorities of the country? Is the Government committed (both in terms of timing and financially) to the project? How is the project aligned with and supports the national, regional and community strategies/plans?
• Has the project involved relevant stakeholders through consultative processes or information-sharing during its preparation phase? Was the human security needs assessment/analysis carried out at the beginning of the project reflecting the various needs of different stakeholders? Are these needs still relevant? Have there any new, more relevant needs emerged that the project should address?

Effectiveness

• How effective has the project been in establishing ownership by the stakeholders? Can the project management and implementation be considered as participatory?

• Is the project making sufficient progress towards its planned objectives/outcomes/outputs? What are the key achievements, challenges and implementation lessons?
• How effectively are the beneficiary interventions in line with actual needs?
• How effective the project addressed the challenges, emerged as a result of the COVID-19 crisis and hostilities in and around NK.

Efficiency

• To what extent the project made good use of the human, financial and technical resources, and have used an appropriate combination of tools and approaches to pursue the achievement of project results in a cost-effective manner?
• Was there a clear distribution of roles and responsibilities of key actors involved?
• To what extent did the project capitalize on other complementary initiatives to the project to reinforce the results of the project?
• Have project funds and activities been delivered in a timely manner? If not, what were the bottlenecks encountered?

Sustainability

• To what extent did the project support the government and beneficiary communities in developing capacities and establishing mechanisms to ensure ownership and the durability of effects in line with Government reforms and strategies such as Community Consolidation, Disaster Risk Reduction, etc?
• What are the sustainability mechanisms used during the Project implementation?  

Impact

• Has the project contributed or is likely to contribute to long-term social, economic, technical, environmental changes for individuals, communities, and institutions in achieving the SDG agenda?
• How the Project contributed to increase the level of human security in targeted communities?

Evaluation questions will be further adjusted and refined by the Evaluator(s) during the desk review phase.

In addition to five main evaluation criteria, the evaluation will review also how project incorporated principles of the human rights-based approach, gender equality aspect and other relevant cross-cutting issues. Gender equality concerns should be integrated in the evaluation scope of analysis and methods and tools for data collection, as well as, should be reflected in evaluation findings, conclusions and recommendations wherever possible.

Cross-cutting issues:

Gender equality and inclusion of the identified vulnerable groups (principle of Leaving No One Behind):

• Assess steps, undertaken to ensure the active and equal participation of women, men, boys, and girls within the programme and advise the steps, required to be taken in the future to improve women participation;
• Did the programme meet specific gender indicators and targets outlined within the project document?
• How were the target  groups (including children, persons with disabilities and youth) , migrants, displaced population, others involved in the project? What impact has the programme had on reducing vulnerabilities of these age groups?

Evaluation methodology

The evaluation methodology will be guided by the Norms and Standards of the United Nations Evaluation Group (UNEG). The evaluation will be conducted in a participatory manner: key stakeholders will be involved in all phases of the evaluation, including the planning, inception, fact-finding and reporting phases.

In this evaluation mixed method approach will be applied by combining qualitative and quantitative components to ensure complementarity. The analysis will be built on triangulating information collected from different stakeholders (Project staff, Project partners, stakeholders, and beneficiaries) through different methods including secondary data and documentation review and primary data. Regular meetings will be organized with the Inter-Agency Working Group, as the main entity for Project implementation, informing on and verifying the stages of evaluation, seeking relevant data and coordinating organizational issues. It should critically examine the information gathered from the various sources and synthesize the information in an objective manner. If contradictory information is obtained from different stakeholders, an effort should be made to understand the reasons for such information, including any gender-based factors and differences.

The Lead Evaluator shall review the following documents before conducting any interviews: Project documentation, progress reports, work plans, monitoring data, workshop reports, country data, policies, legal documents, etc.

Preliminary suggestions for data collection methods to be envisaged include:

• Desk review including review of analysis of existing documents, legal and policy framework; review of monitoring and evaluation reports, available reports and analysis generated through the project;
• Key informant interviews with national and local administrations,  policymakers, community focal points, partners organizations;
• Expert interviews with project implementing agencies;
• Focus group discussions and interviews with beneficiaries, including the identified marginalized groups.

The independent evaluator will finalize the list of key stakeholders/informants (including but not limited to project implementers, decision makers, direct and indirect beneficiaries, etc.), and appropriate data collection methods for each informant category (such as semi-structured or in-depth interviews, expert interviews, focus groups), in close coordination with the project team.

A combination of these methods should be proposed by the independent evaluator in the detailed evaluation methodology.

In close cooperation with the Project Coordination Committee mechanisms will be used to validate the Evaluation findings, conclusions and recommendations also to avoid possible bias  and project team, the independent evaluator will also be responsible for the development of appropriate instruments, including questionnaires, interview and focus group guides, for each of the methods selected. The materials should be gender-sensitive in language and presentation, as well as take into consideration human rights and equity angles.

The  evaluation will be guided by the UN Ethical Guidelines for Evaluation as guiding principle to ensure quality of evaluation process, especially apropos conflict of interest, confidentiality of individual informants, sensitive to beliefs, manners and customs, discrimination and gender equality, to address issues of vulnerable population.

A major limitation to the evaluation will be the potential for impossibility of face-to-face interviews due to COVID-19 restricting measures, thus data will be obtained through online means, though following all strict guidelines to the extent possible.

Evaluation products (key deliverables)

1. Evaluation Workplans and Inception Report: Evaluation methodology, including Evaluation Matrix, data collection tools/questionnaires, list of beneficiaries and stakeholders to be interviewed, draft interview schedules and reports.
2. Draft Evaluation Report:  After the field activities (online interviews conducted), the Lead Evaluator will submit a draft evaluation report for the Project, highlighting initial findings, conclusions, recommendations,  lessons learnt, etc . The Preliminary findings of the Evaluation will be presented as a brief PPT presentation to Project Team for initial discussion and verification.
3. Final evaluation report: Evaluation report with an Executive Summary in English including key recommendations (minimum 30 pages plus annexes). Evaluation report shall be in line with the UN Evaluation Group standard 4.9. It shall be evidence-based, presenting the Project’s progress vis-à-vis the Results Framework, based on triangulated data, findings and recommendations on Project planning, programming, necessary adjustment to the course change, etc.
4. Separate 1-2 pager summary brief with infographics summarizing the key findings of the evaluation for sharing with external audiences.
5. Evaluation methodology, including the Evaluation Matrix, data collection tools/questionnaires, list of beneficiaries and stakeholders to be interviewed. Field mission plans and reports – outlined in an evaluation inception report.
6. Data collection and analysis and draft outline of the Evaluation Report.
7. Evaluation report with an Executive Summary in English including key recommendations (max 30  pages plus annexes). Evaluation report shall be in line with the UN Evaluation Group standard 4.9. It shall be evidence-based, presenting the project’s progress vis-à-vis the Results Framework, based on triangulated data. The report shall present findings and recommendations on project planning, programming, relevance, effectiveness, efficiency, impact and sustainability of interventions.

Related Evaluation Activities

To achieve the objectives and produce the deliverables of the evaluation, the Lead Evaluator will be expected to: 

1. Contextualize the Project interventions.
2. Prepare Inception Report.
3. Conduct meetings (online interviews) with stakeholders

• The UNDP project team will brief the Lead Evaluator and evaluation team and provide all necessary details and clarifications on the documents made available for the document review.
• The evaluation team will have meeting and discussions with the project team, Project Coordinator, UNDP DRR Programme Manager and other unit staff as relevant, UNDP Resident Representative a.i.
• The evaluation team will meet with relevant National and local government representatives, project implementing partners, civil society organizations, etc to learn on their experiences with the project.
• The evaluation team will meet with donor representatives and relevant development partners.

    4. Consultation on draft report and recommendations following the submission of the draft report, undertake consultations with UNDP to      receive feedback for incorporation into the final report.

Evaluation team composition, required competencies and skills for Lead Evaluator:

The Evaluation will be conducted by an independent Lead Evaluator, with the support of the Evaluation assistant The selection of the consultants will be aimed at maximizing the overall “team” qualities. The consultants cannot have participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) and should not have a conflict of interest with project’s related activities. 

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partner.

Implementation arrangements

The evaluation will be conducted by an Evaluation team comprised of independent consultants, including a Lead evaluator and an Evaluation assistant. The team will work in close collaboration with the Project Coordination Team and under the leadership of UNDP as lead agency for the joint project responsible for administering the evaluation. Final evaluation of the consultancies will be implemented by UNDP based on written feedback from all agency representatives of the joint project.

The Project Coordinator and UNDP Results-based Management Programme Analyst will be the main focal points for the day to day communication and administrative matters for the Evaluation team, while regular debriefings will be organized with the Project Coordination Team comprising of technical representatives and M&E focal points from 6 participating agencies to ensure the adherence of the evaluation process to UNEG norms and standards, providing necessary information, quality assurance and guidance for the Evaluation Team. The deliverables of each stage of the evaluation process needs to be reviewed by the Project Coordination Team, prior to further implementation and presentation to the Project Board.

In order to ensure impartiality and national ownership of the evaluation, the key deliverables will be presented and validated through the Project Board gathering high level representatives from line Ministries and six UN agencies, as well as consolidated community and regional administrations, that will serve as the Evaluation Reference Group (ERG). Conducting the evaluation in a participatory manner and involving relevant stakeholders at key stages of the evaluation will also contribute to building evaluation capacity, confidence to evaluation results and further use of the generated recommendations for evidence-based policy-making.

Time frame for the evaluation process:  

*UNDP evaluation process and documents are quality-assessed by the UNDP Independent Evaluation Office (IEO).  Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines -  http://web.undp.org/evaluation/guideline/section-6.shtml

Payment Mode and schedule:

The payment instalments (Inception Report 30%; Final Report 70%) will be made upon satisfactory delivery of all the Deliverables and approval by the Commissioning Unit, Project Coordinator in consultation with Implementing Partners.

Key Documents to review

• Joint Project and Agency Project Document(s)
• Project Mid-Term Evaluation Report
• Results and resources Frameworks
• Project’s Annual Progress Reports
• Project Budgets and Expenditure Reports
• Detailed Workplans and Revisions
• CCA, UNDAF Evaluation, Agency and relevant other evaluations
• UNDAF Results Groups Monitoring and reporting documents
• Monitoring files with analysis of disaggregated data (women, men, boys, girls), data from the field
• Relevant other documents, reports, evaluations, and evidence.

Suggested contents page

Opening pages (acknowledgments, list of acronyms)

Executive Summary (5-6 pages)

Chapter I Background, Object and Methodology

 1.1 Introduction

1.2 Background and context of the project

 1.3 Object of the Evaluation

1.4  Purpose, Objectives and Scope of the Evaluation

1.5 Evaluation Methodology (short)

 1.6 Major Limitations

 1.7 Ethical considerations, Human Rights and Gender

Chapter II Analysis and Findings

2.1 Relevance

2.2 Effectiveness

2.3 Efficiency

2.4 Sustainability

2.5 Impact

2.6 Cross-cutting topics

Chapter III Conclusions and Recommendations

3.1 Conclusions and Lessons Learned

3.2 Recommendations

ANNEXES

1. Terms of Reference
2. Desk Review and Background Documents
3. List of Stakeholders Interviewed
4. Detailed Methodology
5. Interview Guides and Survey Instruments,
6. Output tables

Compétences

• Strong data collection, analysis and writing skills in English;
• Substantive knowledge of concept and principles of local development and governance processes, as well as subject-matter international instruments;
• Strong analytical capacity and creative thinking;
• Proven capacity to write analytical reports;
• Strong planning skills and ability to respect deadlines;
• Excellent communication and oral presentation skills in English;
• Excellent teamwork skills; ability to consult, involve and work with stakeholders of different backgrounds, points of view and interests;
• Demonstrated initiative, high sense of responsibility and discretion;
• High level of integrity, professionalism and respect for diversity.
• Availability to travel as required.

Qualifications et expériences requises

Education:

• Minimum Master’s degree in development studies, law, social sciences,  or similar field. MA in development, law, social science would be considered where experience prevails.

Experience

• 10 years of professional experience in programme/project development, implementation, monitoring and evaluation for the international organizations in the above-mentioned areas; experience in development and application of methodologies for evaluation and assessment, including tools and techniques.
• 5 years of experience in development and application of methodologies for evaluation and assessment, including tools and techniques.
• Experience of leading evaluations and teams
• Proven experience in working with community development projects.

Languages

• Fluency in English and Armenian. Knowledge of Russian is an asset.

CANDIDATES WILL BE EVALUATED BASED ON THE FOLLOWING METHODOLOGY:

Candidates will be evaluated using a cumulative analysis method taking into consideration the combination of the applicants' technical qualifications, experience and financial proposal.The contract will be awarded to the candidate whose offer has been evaluated and determined as technically responsive/compliant/acceptable to the requirements of the ToR and received the highest cumulative (technical and financial) score out of below defined technical and financial criteria.

Only candidates obtaining a minimum of 49 points in the technical evaluation would be considered for financial evaluation.

Technical Criteria - 70% of total evaluation - max. 70 points

Financial Criteria - 30% of total evaluation - max. 30 points.

DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE APPLICATION.

• Financial Proposal and Availability form: OFFEROR’S LETTER TO UNDP (A total lump sum  option is applicable for this assignment)
• CV shall include Education/Qualification, Processional Certification, Employment Records /Experience

General Conditions of Contract is available here: GENERAL CONDITIONS OF CONTRACT

Note: Please note you can upload only one document to this application (scan all documents that should be included in the application in one single PDF file and upload it).