Background

“Integrated Support to Rural Development: Building Resilient Communities” (ISRDBRC) Project is financed by the Government of the Russian Federation and was implemented in bordering communities of Tavush Region of the Republic of Armenia over five years. The overall goal of the project is to ensure balanced development of RA regions through an integrated socio-economic approach, as well as raising the quality of life and income level of the local population in bordering areas of Tavush region. Project budget is  USD 5,251,340.87.

The Project has three main components:

• Development planning. Will include elaboration of participatory integrated community development plans for 45 communities with consideration of socio-economic and environmental, including disaster risk reduction, pillars.
• Community sustainability activities – Agriculture and agro-processing. Will include: sector-specific capacity building, establishment of agricultural machinery pools, construction of small greenhouses, establishment of small-scale contemporary agro-processing units, establishment/improvement of orchards/vineyards, and establishment of collection centers, etc.
• Community sustainability activities – Energy Efficiency/sustainable water management/infrastructure rehabilitation. Will include: rehabilitation of community infrastructures with focus on energy efficiency, introduction of water saving technologies, ensuring access to safe drinking water, etc.

It is supposed that the residents of all 45 bordering communities of Tavush region, with an overall population of 62,000, will get direct or indirect benefits from the project in the form of: access to quality infrastructures, collection center of agricultural products and parks of agricultural machinery, as well as employment in agricultural and agro-processing sectors.

Objective of Assignment:

The final evaluation of the Project is designed to measure results and, if available at this stage already, the impact of the project.

In particular, this external evaluation will focus on evaluating and learning from the project results and lessons throughout the project implementation. The findings and recommendations of the evaluation will inform the future initiatives by UNDP Armenia, the Government of Armenia, and the main stakeholders.

In this context, the evaluation will assess how the project has contributed towards its expected outcome of enabling the economic empowerment of locals through engagement in primary and secondary sustainable agricultural production and improved their livelihoods through facilitating access to rehabilitated infrastructure through participatory community development planning.

The final evaluation will assess how the project achieved its goal of participatory community planning, income generation through agriproduction and processing, and rehabilitated community infrastructures in 45 borderline communities of Tavush region and evaluate the replicability of the interventions.

The results and recommendations will be used by UNDP broadly and by UNDP in Armenia in particular as a basis for future rural development interventions at the national and local levels, in view of the continued cooperation with the Government of Armenia and the main stakeholders.

The independent external evaluation will be conducted by an independent expert. The evaluation will assess the relevance, efficiency, and effectiveness of the project and will provide recommendations regarding the impact of the project. As stipulated in the project document the main stakeholders and, partners of the project is the Ministry of Territorial Administration and Infrastructure (ex RA MTAD) of Armenia.

Duties and Responsibilities

In accordance with UNDP evaluation guidelines[1], the evaluation will assess the performance of ISRDBRC project in achieving its intended outputs and results (contribution or attribution to outcomes) as per the associated CPD theories of change. The evaluation will assess the project’s effectiveness, efficiency, relevance, impact and sustainability. The evaluation will also review and provide the project’s impact on individuals and households in a disaggregated way.

The specific objectives are:

• To assess project’s achieved outputs vis-à-vis the Results and Resources Framework.
• To assess the project’s attribution and contribution to achieved outcomes, indicating also strengths, weaknesses and unexpected results (projects’ by-products).
• Indicate Project’s contribution/attribution to the achievement of Country Programme Document outputs and outcomes.
• To determine the overall efficiency in the utilization of resources in achieving results.
• To assess the appropriateness of the design of the project and the implementation arrangements, including but not limited to the project modality, organizational structure, and coordination mechanisms set up to support the project.
• To assess the extent to which the project has contributed to the creation of an enabling environment, and the extent to which this has helped shape effective government policies and programming on disaster management and risk reduction.
• To assess the sustainability of results and provide recommendations for sustaining the benefits of the project and how to improve sustainability in future initiatives.
• To assess the approach to capacity development and whether initiatives have contributed to sustainability.
• To review the effectiveness of the gender mainstreaming strategy and partnership strategy.
• To gain insights into the level of client satisfaction with the project. The clients include community and local government beneficiaries; national government partners and donors.
• To identify best practices and lessons learned which can be replicated.

The core criteria to be considered in this evaluation are as follows:

• Relevance: the extent to which intended outputs and outcomes of the project are consistent with national and local policies and priorities and the needs of intended beneficiaries.
• Effectiveness: the extent to which the intended results have been achieved and whether opportunities created by the project were equally accessible for women and men.
• Efficiency: how economically resources or inputs (e.g., funds, expertise and time) were converted to results.
• Sustainability: the extent to which benefits of the project continue after external development assistance has withdrawn. This includes evaluating the extent to which relevant social, economic, political, institutional, and other conditions are present and, based on that assessment making projection about the national capacity to maintain, manage and ensure the development results in the future.
• Impact: changes in human development and people’s wellbeing that are brought about by development initiatives, directly or indirectly, intended or unintended. 

EVALUATION CRITERIA. INDICATIVE EVALUATION QUESTIONS

In accordance with UNDP evaluation guidelines, specific questions related to each of criteria can include the following:

Relevance: evaluate the pertinence of project objectives and purposes in relation to the project expected results (impact), target groups, direct and indirect beneficiaries.

1. What is the present level of relevance of the project?
2. Are the project overall objectives consistent with, and supportive of Partner Government policies?
3. Does the project still respond to the needs of the key partners?
4. Are the project objectives and results clear and logical, and do they address clearly identified needs?
5. Are there suitable and informative targets, e.g. are they Specific, Measurable, Achievable, Realistic and Time-bound (SMART)?
6. Are the activities planned appropriately to achieve output(s) and whether the output(s) lead to the expected project outcome?
7. Is the current design sufficiently supported by all stakeholders?
8. Have key stakeholders been involved in the design process?
9. Are coordination, management and financing arrangements clearly defined and do they support institutional strengthening and local ownership?
10. Are the objectives clearly understood by the project partners?
11. If applicable: How well has the project design been adapted to make it more relevant? Was it straightforward to do contractually?
12. Have the relevant cross-cutting issues (environment, gender, human rights and governance, donor coordination or others) been adequately mainstreamed in the project design?
13. Was the project aligned with government and UNDP priorities?
14. Was the project appropriate to the local context?

Effectiveness: evaluate project effectiveness and to what extent has the project produced its desired objectives.

1. How well is the project achieving its planned results?
2. Have the planned results to date been achieved?
3. Are the targets for the project appropriate and are they being reported against?
4. What is the quality of the results/services available?
5. Are there any factors which prevent target groups accessing the results/services?
6. To what extent has the project adapted or is able to adapt to changing external conditions (risks and assumptions) in order to ensure benefits for the target groups?
7. Are the risks and assumptions holding true? Are risk management arrangements in place?
8. To what extent are unplanned positive effects contributing to results produced/ services provided?

Efficiency: evaluate to what degree have resources been optimally used during project implementation, and has the project achieved satisfactory level of cost effectiveness.

1. How well are inputs/resources being managed?
2. To what degree are inputs provided/ available on time to implement activities from all parties involved?
3. To what degree are inputs provided/ available at planned cost (or lower than planned), from all parties involved?
4. Are project resources managed in a transparent and accountable manner?
5. Are all contractual procedures clearly understood and do they facilitate the implementation of the project?
6. How well is the implementation of activities managed?
7. Is an activity schedule (or work plan) and resource schedule available and used by the project management and other relevant parties?
8. To what extent are activities implemented as scheduled? If there are delays how can they be rectified?
9. Are funds committed and spent in line with the implementation timescale? If not, why not?
10. How well are activities monitored by the project and are corrective measures taken if required?
11. If appropriate, how flexible is the project in adapting to changing needs?
12. If appropriate how does the project co-ordinate with other similar interventions to encourage synergy and avoid overlaps?
13. How well are outputs achieved?
14. Have all planned outputs been delivered to date? And in a logical sequence?
15. What is the quality of outputs to date?
16. Are the outputs achieved likely to contribute to the intended results?
17. Are they correctly reflected through the targets?
18. Do the inter-institutional structures e.g. steering committees, technical team meeting and monitoring systems, allow efficient project implementation?
19. Have all partners been able to provide their financial and/or other contributions?

Sustainability: evaluate the contribution to sustainability of benefit streams (to what extent benefits will continue after the life of the project).

1. Is sustainability an integral part of the design i.e. is there a phase out/hand over strategy?
2. Is the sustainability strategy fully understood by the partners?
3. If the services/results have to be supported institutionally, are funds likely to be made available? If so, by whom?
4. Are the services/results affordable for the key partners at the completion of project?
5. What is the level of ownership of the project by key partners and will it continue after the end of external support?
6. How far the project is embedded in local structures?
7. To what extent are relevant key partners actively involved in decision-making concerning project orientation and implementation?
8. What is the likelihood that key partners will continue to make use of relevant results?
9. Do the key partners have any plans to continue delivering the stream of benefits and if so, are they likely to materialise?
10. What is the level of policy support provided and the degree of interaction between project and policy level?
11. What support has been provided from the relevant national, sectoral and budgetary policies?
12. Do changes in government policies and priorities affect the project and how well is it adapting in terms of long-term needs for support?
13. Are the material, services and equipment support likely to continue after the project has finished?
14. How well is the project contributing to institutional and management capacity?
15. What lessons can be drawn from the coordination efforts and working arrangements between the project team, its counterparts/beneficiaries, and partner organizations?

Impact: evaluate the project impact, if available at this early stage.

1. What are the direct impact prospects of the project on individuals and households in a disaggregated way?
2. Are any external factors likely to jeopardize the project’s direct impact?
3. To what extent does/will the project have any indirect positive and/or negative impacts? (e.g., social, cultural, gender, economic)
4. Have there been/will there be any unplanned positive impacts on the planned key partners or other non-targeted communities arising from the project? How did this affect the impact?
5. Did the project take timely measures for mitigating the unplanned negative impacts? What was the result?

Recommendations, lessons learned and best practices

Provide key recommendations related to the project design, implementation, management, management of resources, programmatic response.

• What lessons can be learned from the project implementation in order to improve performance, result and effectiveness in the future.

EVALUATION METHODOLOGY

The evaluation will be conducted through a qualitative assessment method. The evaluation phases shall include, but not be limited to:

• A desk review of relevant reports and data that will mainly address qualitative issues.
• Submission to and discussion of the proposed methodology with UNDP (inception report)
• Field-research and visiting partners and beneficiaries.
• Preparation of the evaluation report, findings and recommendations.
• Review findings with stakeholders/partners and preparing a follow-up action plan to implement accepted recommendations

Consistent with the above general scope, the Consultant will work closely with CO Evaluation Manager, Socio-Economic Portfolio Manager, ISRDBRC Project Manager and the team, particularly: 

Submission/discussion of Evaluation Methodology

• Evaluation methodology should be submitted and discussed with/approved by UNDP Senior Management, CO Evaluation Manager, Socio-Economic Portfolio Manager, ISRDBRC Project Manager.

Desk Review

During the desk review, the written material that should be examined may include but not be limited to:

• The original Project Document and any subsequent costed work-plans;
• Project reports with key budgetary information;
• Minutes and conclusions of steering committee meetings;
• Summaries of the participatory processes;
• Information on the activities of project implementation team;
• Project’s communication materials
• Any other material that would deem relevant.

Field Visit

• Online and/or where possible face-to-face interviews with the stakeholders, including members of the project implementation team. The evaluation team should provide, some days in advance of their visit, a note summarizing those issues that they would particularly look to explore further and a proposed schedule.
• Discussions with the key partners, target audience, and relevant stakeholders.

Presentation of Results, Reporting and Final Submission

• The final output of the evaluation will be a comprehensive report in UNDP format outlining the methodology pursued and main findings of the evaluation, including lessons learned and recommendations. The findings of the evaluation will be presented by the evaluator to UNDP as well as to the Project Board for review and inputs. Inputs will be integrated final evaluation report will be submitted to UNDP on the date agreed.

Terms and Conditions for provision of services:

The assignment will be a combination of in country missions and the desk study. Site visits or online evaluation modality will be discussed to avoid any risks of COVID-19 for interviewer and interviewees. 

Expected result:

Submit the expected written outputs above in printed and electronic versions; MS Word (.doc) format including MS PowerPoint (.ppt) presentation if necessary.

Evaluator Ethics

Evaluation consultants will be held to the highest ethical standards and are required to sign a Code of Conduct (Annex E) upon acceptance of the assignment. UNDP evaluations are conducted in accordance with the principles outlined in the UNEG 'Ethical Guidelines for Evaluations'.

Evaluation timeframe:

The total duration of the evaluation will be 23 days according to the following plan:

Evaluation deliverables

The evaluator is expected to deliver the following:

*When submitting the final evaluation report, the evaluator is required also to provide an 'audit trail', detailing how all received comments have (and have not) been addressed in the final evaluation report.

Payment mode:

100% of the payment will be made upon effective conclusion of the Deliverables and submission and approval by UNDP.

 Annex 1: The Report should include but not be limited to the following headings Title and opening pages

• Table of contents
• Introduction
• Description of the intervention
• Evaluation Scope and objectives
• Evaluation approach and methodology
• Data analysis, Findings and conclusion
• Recommendations
• Lessons learned
• Annex(s), including interview questions

[1]  For detailed information refer to the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results (pages 168-170):  http://www.undp.org/evaluation/handbook

Competencies

• Demonstrated experience with project/programme assessments, evaluations;
• Proficiency in monitoring and evaluation techniques including in-depth interviews; focus group discussion and participatory information collection techniques;
• Strong analytical capacity;
• Advanced experience in working with government agencies (central and local), civil society organizations and international organizations;
• Understanding of country community development and consolidation context in Armenia is an asset;
• Advanced communication capacity;
• Ability to work efficiently and provide high quality outputs under time pressure;
• Advanced IT and Microsoft Office operating capacities.

Required Skills and Experience

Education: 

• Advanced Education in relevant field (e.g., Economics, Social Science and Development, Business Administration, International Development, etc.)

Experience:

• At least 5 years of proven experience in project evaluation under UNDP and/or other international organizations in the area of integrated rural development, area development, or community development;
• Proven practical experience in information gathering and research methodology; experience in data analysis by using innovative approaches and goof writing of prompt recommendations.   

Languages:

• Proficiency in the Armenian and English languages are required. The Evaluation report should be submitted in English, while the interaction with beneficiaries and stakeholders should be done in Armenian.

CANDIDATES WILL BE EVALUATED BASED ON THE FOLLOWING METHODOLOGY:

Candidates will be evaluated using a cumulative analysis method taking into consideration the combination of the applicants' technical qualifications,  experience and financial proposal. The contract will be awarded to the individual consultant whose offer has been evaluated and determined as technically responsive/compliant/acceptable to the requirements of the ToR and received the highest cumulative (technical and financial) score out of below defined technical and financial criteria.

Only candidates obtaining a minimum of 49 points in the technical evaluation would be considered for financial evaluation.  

Technical Criteria - 70% of total evaluation - max. 70 points

Financial Criteria - 30% of total evaluation - max. 30 points. 

DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE APPLICATION.

• Financial Proposal and Availability form: OFFEROR’S LETTER TO UNDP  
• CV shall include Education/Qualification, Processional Certification, Employment Records /Experience

General Conditions of Contract is available here: GENERAL CONDITIONS OF CONTRACT

Note: Please note you can upload only one document to this application (scan all documents in one single PDF file to attach).