Background

The Integrated Territorial Development 2 project has built on the successes of the Austrian Development Cooperation-funded UNDP “Local-level Response for Employment Generation and Integrated Territorial Development (INTERDEV)” project that was implemented in southern Kosovo’s municipalities of Dragash/Dragaš and Shtërpcë/Štrpce since 2014. This project has proven to address some of the paramount challenges citizens of not only the partner areas, but of Kosovo in general, face: high rate and long spells of unemployment, lack of decent income generation, poor public services, as well as socioeconomic exclusion of women, youth, and other vulnerable groups.
The INTERDEV phase 1 provided over 900 local citizens with jobs and additional income flow, established 3 social enterprises that not only employ vulnerable individuals of the local societies, but also add value to agricultural and textile production in the area, upgraded 325 rural micro and small enterprises which have increased their income and created jobs for local communities, developed capacities of municipal representatives, private sector and the civil society in topics of economic development and agriculture, and empowered local officials with improved understanding of effective socioeconomic development and inclusion, importance of human-rights based, participatory processes, and appropriately responding to citizens’ needs. A key element of the intervention, the INTERDEV stimulates the potentials of local contexts and harnesses local ownership and leadership over the project’s services.
The excellent example of the close cooperation with partner municipalities is worth highlighting, where they have claimed full ownership and leadership of project implementation, monitoring, and results dissemination. By doing so, the municipalities have demonstrated to be very inclusive in approach by bringing to the process all relevant stakeholders in their municipalities. Local Action Groups have played a crucial role in the positive outcome of the current phase of INTERDEV and the municipal partners have strengthened their capacities and understanding of the local economic development and the role they should play in public service delivery. 
The INTERDEV 2 has continued to utilize tested methodologies and, as a successful approach, has scaled up its services to a new municipality of Viti/Vitina in the south-east of Kosovo. With the overall objective that inclusive and sustainable income generation and job creation for women and men is improved in the municipalities of Dragash/Dragaš, Shtërpcë/Štrpce, and Viti/Vitina, the project’s purpose is to enhance municipal public service provision in rural development and, in a gender equitable, socially inclusive and environmentally sustainable manner, expand economic activity of local micro and small businesses. It is expected that as a result of project’s three core pillars of activities: 1) municipal officials have enhanced capacities in provision of services in rural development; 2) local micro and small enterprises and farmers have been supported to upgrade their businesses; and 3) bottom-up approaches and local-level concertation for employment generation in the form of Territorial Employment Pacts operate at the municipal level.
Working closely with a variety of public, private sector, and non-governmental sector partners on the local level (such as the Local Action Groups, local Employment Offices, Vocational Training Centres, the Local Development Fund, and the private sector companies in the area), the INTERDEV 2 has continued to serve citizens who are at risk of socioeconomic exclusion: local smallholder farmers, micro business initiatives, long-term unemployed and low-skilled or unskilled residents, women, youth, non-majority communities, as well as people with disabilities. As such, the project in its three years aimed to serve over 700 additional residents (50% women, and 40% non-majority communities) and reach over 3,800 indirect beneficiaries. More details about the project are presented in the following section.
The February 2017 – January 2020 INTERDEV phase 2 entered its final stage of implementation and an external final evaluation of the project activities will be conducted to assess the overall progress of the project towards the expected results, and provide lessons learned and recommendations for future interventions of similar nature. To this end, the project will hire an Evaluation Team composed of an international evaluation specialist and a local evaluation specialist, who will work jointly to achieve the expected results. Moreover, in second half of 2018, the project underwent an external mid-term evaluation, and this final evaluation will also identify how the mid-term evaluation recommendations were addressed through the implementation of the project activities in its second half.

Objectives of Assignment
The overall objective of this assignment is to conduct a final evaluation of the project in terms of its overall relevance, impact, effectiveness, efficiency, and sustainability, and elaborate on the lessons learned and recommendations for future interventions of similar nature. The scope of the evaluation is dedicated to the INTERDEV phase 2 and should look both into the output-level results as well as the outcome-level results as key indicators of the overall project performance. Where relevant, the evaluation team should consider the INTERDEV phase 1 as well, in particular when it comes to changes from phase 1 to phase 2.

Project Information
Project/outcome title: Integrated Territorial Development 2 (INTERDEV 2)
Atlas ID: 00079191/00104334 & 00104340
Corporate outcome and output:
UNDP SP 2018-2021 (Outcome 1: Advance poverty eradication in all its forms and dimensions; Output 1.1.2 Marginalized groups, particularly the poor, women, and people with disabilities and displaced are empowered to gain universal access to basic services and financial and non-financial assets to build productive capacities and benefit from sustainable livelihoods and jobs)
Kosovo CPD 2016-2020 (Outcome 2. Education and employment policies and programmes enable greater access to decent employment opportunities for youth and women; Output 2.1. Implementation of territorial employment pacts at local level)
Country: Kosovo (as per UNSCR 1244/1999)
Region: Western Balkans, ECIS
Date project document signed: 08 February 2017
Project dates: Start: 01 February 2017, Planned end: 31 January 2020
Project budget: 1,995,000 EUR (Original budget 1,955,000.00 EUR + 40,000 EUR additional funds from municipalities)
Project expenditure at the time of evaluation: 1,294,767.58 EUR
Funding source: Austrian Development Cooperation, Third Parties (Municipalities, MLSW), UNDP
Implementing party : UNDP, and Local Development Fund for Output 00104340

Duties and Responsibilities

The consultant will work together with and lead the Local Specialist under direct supervision of the Project Manager, in close consultation with the Programme Team. The project team will provide administrative and logistical support as needed. In order to achieve the above objective, the main tasks of the International Specialist (as the leader of the Evaluation Team) is to:

  • In close cooperation with the Local Specialist, conduct a comprehensive desk review of relevant project-related documents and UNDP evaluation policies and, based on this information, draft and submit an inception report with appropriate methodology to be applied during the evaluation, the evaluation matrix, as well as the work plan and any technical instruments to be used during the course of the assignment, while being guided by the set of evaluation questions as presented below (4 w/ds);
  • Together with the Local Specialist, conduct on-site field visits, meetings, discussions, and interviews with relevant stakeholders and project beneficiaries in Kosovo. The Evaluation Team is expected to share the list of interviews to be conducted beforehand, and receive feedback and clearance from UNDP and the Austrian Development Agency (ADA). An initial briefing meeting with the UNDP team will be held in order to finalize the evaluation design (7 w/ds);
  • Jointly with the Local Specialist, hold a debriefing workshop at the end of the mission with main stakeholders to summarize initial findings and recommendations (1 w/d);
  • Supported by the Local Specialist and based on the feedback received during the debriefing workshop, draft a final evaluation report containing the methodology applied, a presentation of findings, presentation of the lessons learned and clear strategic recommendations to the UNDP and its partners for future interventions of similar nature in the target areas and beyond. These recommendations should contain specifically to whom of each of the partners of the project they are addressed. The International Specialist is the lead writer of the evaluation report. (7 w/ds);

The final evaluation report must include, but not necessarily be limited to, the elements outlined below:

  • Results Assessment Form of ADA (to be provided)
  • Title and opening pages
  • Table of contents
  • List of acronyms and abbreviations
  • Executive summary, including a summary of the lessons learned and recommendations
  • Introduction
  • Description of the intervention
  • Evaluation scope and objectives
  • Evaluation methodology
  • Data analysis
  • Findings and conclusions 
  • Lessons learned
  • Recommendations  
  • Report annexes

Finalize the final evaluation report, accounting for the Austrian Development Agency, UNDP and stakeholders’ feedback on the first draft (3 w/ds).

The following evaluation criteria are to be used as per the UNDP methodology, and related evaluation questions are proposed for the evaluation process; however, these can be expanded, prioritized, and modified by the evaluator during the inception phase in consultation with UNDP and ADA. Each evaluation criteria must be ranked as per UNDP ranking methodology that will be shared with the Evaluation Team during the inception phase of the assignment.

Evaluation questions:

Relevant evaluation criteria

Key questions suggested

Relevance

  • Are/Were the project’s activities relevant for the main beneficiaries? Has the initiative tackled key challenges and problems identified?
  • To what extent have the cross-cutting issues (such as environment, gender equality, women empowerment, human rights based approach and social standards), principles and quality criteria (i.e. for private sector development[1]) of the Austrian Development Cooperation been duly considered/mainstreamed in the project implementation and how well is this reflected in the project reports? How could they have been better integrated?
  • Have there been any changes in policies and strategy development that have affected the project? If yes, have necessary revisions and adaptations been designed? To what extent has the project been appropriately responsive to political, legal, economic, institutional, etc., changes in Kosovo?
  • How did the project link and contribute the national development priorities, the ADA and UNDP Kosovo programme’s outputs and outcomes, the UNDP Strategic Plan and the SDGs?
  • How did the project contribute to the theory of change for the relevant UNDP Kosovo programme outcome?
  • Were perspectives of those who could affect the outcomes, and those who could contribute information or other resources to the attainment of stated results, taken into account during the project design processes?
  • What are the areas of relevance for future interventions in the target area?

 

Effectiveness

  • To what level has the project reached the project purpose and the expected results as stated in the project document (logical framework matrix)?
  • In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
  • In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?
  • What factors contributed to effectiveness or ineffectiveness?
  • To what extent did the project contribute to the Kosovo programme outcomes and outputs, the SDGs, the UNDP Strategic Plan and national development priorities?
  • What challenges have been faced? What has been done to address the potential challenges/problems? What has been done to mitigate risks?
  • In what ways did the project respond to the findings/recommendations of the mid-term evaluation in the second half of project’s implementation?
  • To what extent has the UNDP partnership strategy been appropriate and effective, and to what extent have stakeholders been involved in project implementation?
  • To what extent were project management and implementation participatory and is this participation contributing towards achievement of the project objectives?
  • To what extent has the project been appropriately responsive to the needs of the national constituents and changing partner priorities?
  • To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights?

 

Efficiency

  • Have the resources been used efficiently? How well have the various activities transformed the available resources into the intended results in terms of quantity, quality and timeliness? (in comparison to the plan)
  • Was the overall aid coordination properly ensured in the field of local economic and rural development in the target area?
  • Were the management and administrative arrangements sufficient to ensure efficient implementation and generating the expected results of the project? To what extent has there been an economical use of financial and human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
  • Were the project funds and activities delivered in a timely manner?
  • To what extent do the M&E systems utilized by UNDP ensure effective and efficient project management?

 

Sustainability

  • How has the project ensured sustainability of its results and impacts (i.e. strengthened capacities, continuity of use of knowledge, improved practices, etc.)?
  • Did the project have a concrete and realistic exit strategy to ensure sustainability and what could be done to strengthen exit strategies and sustainability?
  • Are there any jeopardizing aspects that have not been considered or abated by the project actions? In case of sustainability risks (financial, social economic), were sufficient mitigation measures proposed?
  • Has ownership of the actions and impact been transferred to the corresponding stakeholders? Do the stakeholders / beneficiaries have the capacity to take over the ownership of the actions and results of the project and maintain and further develop the results?
  • To what extent will financial and economic resources be available to sustain the benefits achieved by the project?
  • Do the legal frameworks, policies and governance structures and processes within which the project operates pose risks that may jeopardize sustainability of project benefits?
  • To what extent do mechanisms, procedures and policies exist to allow primary stakeholders to carry forward the results attained on gender equality, empowerment of women, human rights and human development?
  • To what extent do stakeholders support the project’s long-term objectives?
  • To what extent the lessons learned were kept and documented by the project team on a continual basis and shared with appropriate parties who could learn from the project?

 

Impact

  • Is there evidence of long-lasting desired changes, in which aspects?
  • How much did the project lead to a change of behaviours and motivations (of local governments) in terms of paying attention to marginalised and vulnerable population groups? Is there clear evidence for it?
  • Has the project appropriately reached its target groups? Did the project serve the needs of vulnerable groups, i.e. women, youth, non-majority communities?
  • What effects were realized in terms of social inclusion? Did vulnerable groups had the same possibilities to participate and benefit, or was there a clear distinguishment?
  • How did the project contribut to (more) sustainable management of natural resources?

 

Stakeholders and Partnership Strategy

  • How has the project implemented the commitments to promote local ownership, alignment, harmonization, management for development results and mutual accountability?

Evaluation

  • Were intended results (outputs, outcomes) adequately defined, appropriate and stated in measurable terms, and are the results verifiable?

Theory of Change or Results/Outcome Map

  • Was the Theory of Change or project logic feasible and was it realistic? Were assumptions, factors and risks sufficiently taken into consideration?

Human rights

  • To what extent have poor, minority groups, physically challenged, women and other disadvantaged and marginalized groups benefited from the project?

Gender

  • To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?
  • Is the gender marker data assigned to this project representative of reality?
  • To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

 

 

[1] Found at: http://www.entwicklung.at/uploads/media/IV_Quality_Criteria_on_Private_Sector_Development_01.pdf

 

 

 

Methodology
Methodological approaches may include some or all of the following. The final methodological approach including the interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, ADA, stakeholders and the evaluators.

  • Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments.

1. Document review of all relevant documentation. This would include a review of inter alia

  • Project document (contribution agreement).
  • Theory of change and results framework.
  • Programme and project quality assurance reports.
  • Annual workplans.
  • Activity designs.
  • Consolidated periodic and annual reports.
  • Highlights of project board meetings. 
  • Technical/financial monitoring reports.

2. Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organizations, and implementing partners:

  • Development of evaluation questions around relevance, effectiveness, efficiency, impact and sustainability and designed for different stakeholders to be interviewed.
  • Key informant and focus group discussions with men and women, beneficiaries and stakeholders.
  • All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.

3. Surveys and questionnaires including participants in development programmes, and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels.
4. Field visits and on-site validation of key tangible outputs and interventions.
5. The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries.
6. Other methods such as outcome mapping, observational visits, group discussions, etc.
7. Data review and analysis of monitoring and other data sources and methods.

  • Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources.

Expected Outputs and Deliverables

  • Inception report (approx. 30 pages) containing appropriate methodology to be applied during the final evaluation, as well as the work plan and technical instruments to be used during the course of the assignment is drafted, submitted, and endorsed by UNDP after consultation with ADA (4 w/days)
  • Field visits, meetings and interviews in Kosovo are conducted, gathering data to be used in the final evaluation report (7 w/days)
  • A debriefing workshop with key stakeholders is held and initial findings and recommendations presented (1 w/day)
  • Draft final Evaluation report with the methodology applied, a presentation of findings, a presentation of the lessons learned and clear strategic and operational recommendations to the UNDP, ADA, and its partners is formulated, based on the findings acquired during the field mission to Kosovo and through the relevant project documentation, and submitted (7 w/days)
  • A finalized Final Evaluation report accounting for the UNDP, ADA, and stakeholders’ feedback on the first draft is produced, submitted to, and validated by UNDP. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments (Evaluation report audit trail) (3 w/days)

Implementation Arrangements

The Evaluation Team Composition

  • The Evaluation Team will be composed of an international evaluation specialist and a local evaluation specialist, who will work jointly to achieve the expected results.
  • The international evaluation specialist is the lead of the evaluation team and the lead author of the evaluation.
  • The consultant will work together with the local specialist under direct supervision of the INTERDEV Project Manager, in close consultation with the UNDP Programme Team. The project team will provide administrative and logistical support as needed.

Evaluation arrangements

  • The INTERDEV Project Manager will provide necessary information for the evaluation, will lead the logistical support of the evaluation (support in arrangements of meetings, field visits), and will be the primary point of contact for the evaluation team.
  • The INTERDEV Municipal Project Support Officers located in each of the three partner municipalities will provide on-site logistical support (support in arrangements of on-site meetings, support in field visits to beneficiaries).
  • The UNDP Programme Team will perform quality assurance of the evaluation process and its outputs.
  • The Project Board will be the recipient of the evaluation findings and provide any feedback, including through the debriefing workshop at the end of the field mission.

Evaluation Ethics

  • This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

Provision of Support Services

  • Office space: No                           
  • Equipment (laptop, etc.): No          
  • Secretarial Services: Yes - responsible INTERDEV 2 team members

Remuneration - Lump Sum Amount:

The Contract is based on lump sum remuneration and shall be processed subject to deliverables as per the schedule listed below: 

  • Upon signature of the contract: 20% of the total amount of the contract
  • Draft Final Evaluation report received: 50% of the total amount of the contract
  • Final version of the Final Evaluation report received and validated: 30% of the total amount of the contract

Recommended Presentation of Offer
The following documents must be submitted in order to be evaluated and considered for the assignment:

  • P11 (signed), indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the candidate and at least three (3) professional references
  • Technical proposal, a max. 2-page document briefly outlining the methodology envisaged for the assignment for delivering the expected results within the indicated timeframe (an interview will be conducted for the shortlisted candidates);
  • Letter of Confirmation of Interest and Availability;
  • Financial proposal, the consultant is expected to provide an all-inclusive lump sum amount financial proposal. The Offeror must indicate at this point and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP. The contract price is fixed regardless of changes in the cost components.

Criteria for selection of the Best Offer
Offers will be evaluated utilizing a combined Scoring method – where the qualifications, technical proposal, and the interview will be weighted a max. of 70%, and combined with the price offer which will be weighted a max of 30%.

Competencies

Corporate Competencies :

  • Committed to highest regards of professionalism, impartiality, accountability, transparency, ethics, and integrity;
  • Displays cultural, gender, religion, race, nationality, ethnicity, and age sensitivity and adaptability;
  • Demonstrates substantial experience in gender equality and social inclusion.
  • Treats all people fairly without favouritism.

Functional Competencies:

  • Ability to work effectively within a team and develop good relationships with counterparts and stakeholders;
  • Ability to synthesise research and draw conclusion on the related subjects;
  • Ability to pay attention to details;
  • Demonstrates transparency and provides feedback to all those who will contribute to the evaluation;
  • Excellent interpersonal skills and ability to communicate effectively, both orally and in writing;
  • Ability to establish effective working relations in a multicultural team environment;
  • Commitment to accomplish work;
  • Responds positively to critical feedback;
  • Results and task oriented.

Required Skills and Experience

Education:
Master’s degree in social sciences, economic development or other related qualification is required.
Experience:
At least 5 years of demonstrated relevant work experience with evaluation of development interventions at national and/or international level is required.
Experience with local economic and rural development is considered a distinct asset. Previous work experience in the Western Balkans, preferably Kosovo in particular, is considered an asset.
Extensive knowledge of results-based management evaluation, as well as of participatory M&E, methodological and practical considerations in conducting evaluations of development interventions is required.
Language requirements:
Excellent spoken, analytical and report writing skills in clear and fluent English are required. Knowledge of Albanian and / or Serbian is considered an asset.

Application Instructions:

  • Click on the "Apply now" button;
  • Input your information in the appropriate sections: personal information, language proficiency, education, resume and motivation; you can type in, or paste your short Resume into the last box;
  • Upon completion of the first page, please hit "submit application" tab at the end of the page. On the next page, you will be asked to upload your Resume;
  • System will only allow one attachment. All docs (CV; P11; financial offer; list of similar profiles should be included as one attachment).
  • Please make sure to submit all the requested documents/information; otherwise, your application will be considered incomplete.