Background

UNDP and ILO jointly implement the ADA funded sub-regional project Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase II. The project aims to strengthen capacities of national institutions and local service providers to design and implement inclusive policies and programmes for individuals at risk of social and labour market exclusion. To achieve these objectives, the project envisages a set of technical assistance and capacity development activities that assist labour market intermediaries in introducing innovative practices of service provision by public employment offices (PES) and centres for social work (CSW). Moreover, the project fosters peer-learning and in-country and regional dialogue on the topic of inclusive employment. In this respect, the project forges partnership with private sector and civil society organizations. This three-and-a-half-year project (December 2018 - June 2022) is being implemented in all six Western Balkan countries/territory i.e., Albania, Bosnia and Herzegovina, Kosovo*, Montenegro, North Macedonia and Serbia. The Total value of the project is  2,268,109.20 EUR (Annex 5).

The project builds on the results of the Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase I project, also implemented jointly by UNDP and ILO in the period 1 May 2016 – 31 December 2018 and it also incorporates the recommendations provided in the final project evaluation.

The project’s theory of change (Annex 4) is built on four major pillars:

1. Collaborative service provision coordinated by central PES and CSW and implemented by local branches is a method to tackle the multiple forms of exclusion of those who are most distant from the labour market,
2. Multi-stakeholder territorial employment partnerships, which mobilize private sector, local governments, academia are best positioned to identify the potentials of local economy and to identify the specific vulnerabilities as well as to co-design suitable solutions and financing models for accelerated jobs creation and employment.
3. Mutual learning and the availability of an inclusiveness facility are effective methods of convergence, as well as design and replication of effective solutions for more inclusive service delivery and practices among public employment services.
4. Mutual learning and the availability of an inclusiveness facility prepare the target countries/territory to adopt the same institutional set up that will be subsumed under the OMC on employment and social inclusion policies of the EU.

Under this structure, the project consists of four components (outputs):

Output 1: PES and CSWs have institutionalized models of integrated and user-centred provision of services for marginalized groups; (UNDP)

Output 2: A mutual learning programme among public employment services (modelled on the EU’s MLP and focused on inclusiveness approaches) is delivered; (ILO)

Output 3: Local multi-stakeholder partnerships have capacities and resources to develop and implement inclusive employment initiatives. (UNDP)

Output 4: An Inclusiveness Facility for public employment agencies to implement inclusiveness adjustments to their practices (services and programmes) is established. (ILO)

As a result of these outputs, this project expects to:

• Increase in capacity for local and national stakeholders to provide customised services for disadvantaged people.
• Increase in capacity for local stakeholders to engage in partnerships aiming to enhance vulnerable groups’ access to formal employment opportunities.
• Institutionalization of pilot innovative models of partnerships for inclusive employment that involve public and business sector and the civil society
• An intensification of the mutual learning opportunities on inclusiveness for practitioners of the public employment services and ministries in charge of employment, as a sustainable modality to improve policies.
• Implementation of a number of short- and medium-term changes and adjustments in the practices of public employment services with respect to making them more inclusive.

Following the outbreak of COVID-19 pandemic, both UNDP and ILO ought to adjust the focus and the modalities of implementation of project activities. While the overall focus of the project has been kept a new logframe and targets have been developed and approved by ADA.

Evaluation Scope:

This final project evaluation of Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase II (January 2019 - June 2022) project will be conducted at the end of project implementation and will cover the entire duration of the project from 1 December 2018 until 30 June 2022.

The evaluation is scheduled between May 2022 and June 2022.

The evaluation includes a data collection, with a field mission to two selected countries out of six that have been involved in project implementation. The evaluation shall cover all aspects of the project including inter relations between the various project components but it will focus on in depth analysis of two out of four project components (outputs) - Output 1: PES and CSWs have institutionalized models of integrated and user-centred provision of services for marginalized groups, implemented by UNDP and Output 4: An Inclusiveness Facility for public employment agencies to implement inclusiveness adjustments to their practices (services and programmes) is established, implemented by ILO.

Evaluation Purpose:

The main purpose of the evaluation is to assess the programmatic progress and performance of the above-described intervention from the point of view of relevance, effectiveness, and sustainability.

The evaluation will assess the project performance and address the following questions with the results and evidence that is available to date.

The findings of the evaluation will contribute to effective programming, organizational learning and accountability. The findings of the evaluation will moreover be used to engage policy makers and other stakeholders at national and local levels in evidence-based dialogues and to advocate for inclusive employment strategies to promote inclusive local and national labour market development with a particular focus on disadvantaged individuals. The evaluation should also provide specific recommendations as to the priority areas that should be considered in next projects implemented by UNDP and ILO, including interventions that require continued support, successful interventions for expansion, and recommendations on prioritizing interventions to maximize impact. It should also define recommendations to improve project management structure. 

The evaluation will follow a participatory approach that will include both implementing agencies management structure, key partners will be represented, and additional consultation conducted with key stakeholders, governmental representatives from relevant ministries and national institutions, with civil society representatives as well as key donor partners.

Evaluation Objectives:

The overall objective of the project evaluation will be to assess the extent, to which the project achieved its overall objectives and outputs as identified in the project document and annual working plans. More specifically, the evaluation will aim to:

• Review effectiveness of the overall Project interventions, its main achievements and overall impact of the poject in improving the inclusivness of the labour market in the Western Balkan;
• Review and evaluate the extent to which project outputs have reached the intended beneficiaries;
• Assess the sustainability of project outputs and benefits - analyze how far has the project ensured the sustainability of the project benefits;
• Asses how the intervention and its results relate and contribute to the achievement of 2030 Agenda for Sustainable Development, EU’s “Credible enlargement perspective for and en-hanced EU engagement with the Western Balkans”, and RCC’s South East Europe Strategy 2020.
• Identify lessons learnt from the project implementation and the interventions in the area;
• Identify future similar needs.

Evaluation Approach and Methodology:

The evaluator is expected to consult all relevant sources of information, such as the project document, project reports, project budget, progress reports, project files and any other material that may be consider useful for evidence-based assessment, including interviews and focus groups with stakeholders and beneficiaries.

The evaluation should provide as much gender disaggregated data as possible.

The evaluators are expected to follow a participatory and consultative approach ensuring close engagement with the project counterparts at the national and local level, Project Board, project beneficiaries and key stakeholders. All relevant project documentation will be made available by the implementing agencies, UNDP and ILO, regional and country/territory.

Considering the mandates to incorporate human rights and gender equality in all UN work these dimensions will require special attention for this evaluation and will be considered under each evaluation criterion. When collecting data, evaluators need to ensure that women and disadvantaged groups are adequately represented.

It is expected that the evaluation consultant will develop an evaluation matrix, which will relate to the below questions (and refine them as needed), the areas they refer to, the criteria for evaluating them, the indicators and the means for verification as a tool for the evaluation. Final evaluation matrix will be approved in the evaluation inception report.

The Consultant is expected to explore evidence and where possible validate the achievement against set targets. The Project’s Logframe Matrix will be used as a benchmark for the evaluation.

The results of this evaluation will be shared with the Project Board, ADA, UNDP and ILO offices and national stakeholders. Information specifically targeting the successes and failures of the project is especially sought after.

The project evaluation will be carried out in accordance with UNDP’s[1] and ILO’s [2] evaluation policies and ADA’s[3] guidelines for project evaluations.

Evaluation Criteria and Questions:

Due to the limited scope of the project, the evaluation should refer and use relevance, effectiveness, and sustainability as the evaluation criteria, mainstreaming gender equality and social inclusion perspective as a cross cutting issues.

More specifically, the evaluation will address the following evaluation questions that will be further refined once the evaluation team is recruited[4]:

   Relevance:

• To what extent was the design of the intervention and its results relevant to the needs and priorities of the beneficiaries? Was the choice of interventions relevant to the situation of the target group?
• To what extent key national partners were involved in programme’s conceptualization and design process?
• Review how the project addressed countries’ priorities. Review country ownership. Was the project concept in line with the national sector development priorities and plans of the country?
• To what extent has gender and human rights principles and strategies been integrated into the project design and implementation?
• To what extent is the intervention aligned with Agenda for Sustainable Development, EU’s “Credible enlargement perspective for and en-hanced EU engagement with the Western Balkans”, and RCC’s South East Europe Strategy 2020?
• To what extent was the design of the intervention relevant to the inclusiveness agenda in the Western Balkan?

           Effectiveness:

• To what extent have the expected results of the project been achieved on both outcome and output levels?
• What are the reasons for the achievement or non-achievement of the project results? Has project achieved any unforeseen results, either positive or negative? For whom? What are the good practices and the obstacles or shortcomings encountered? How were they overcome?
• How effective have the selected programme strategies and approaches been in achieving programme results?
• How does the project fit in and align with other relevant initiatives supported by bilateral and multilateral donors and organizations?
• To what extent the project interventions have ensured coherency in the approach with relevant interventions implemented in the Western Balkans?
• How well did the intervention succeed in involving and building the capacities of rights-holders, duty-bearers, as well as the project partners?
• Have the addressed beneficiaries been reached? What are the achievements in terms of employment and job creation for single individuals and facilitating access to more efficient and effective public services – quantitative and qualitative?
• How the intervention impacted employment outcomes of particularly disadvantaged individuals?
• To what extent are the programme approaches and strategies are innovative for achieving labour market inclusion and gender equality in project countries? What -if any- types of innovative good practices have been introduced in the programme?
• What contribution are participating UN agencies making to implementing global norms and standards for economic empowerment and labour market inclusion?
• To what extent the project improved communication, coordination and information exchange within the national labour markets at all level?
• Is there a clear understanding of roles and responsibilities by all parties involved?

         Sustainability:

• What is the likelihood that the benefits from the project will be maintained for a reasonably long period of time after the project phase out? 
• To what extent the intervention succeeded in building individual and institutional capacities of rights-holders and duty-bearers to ensure sustainability of benefits and more inclusive practices to local development and good governance?
• How effectively has the project generated national ownership of the results achieved, the establishment of partnerships with relevant stakeholders and the development of national capacities to ensure sustainability of efforts and benefits?
• What steps were taken to develop and/or reinforce the operating capacities of national partners during the implementation of the programme?
• To what extent has the project been able to promote replication and/or up-scaling of successful practices from the Inclusive Labour Market Phase I?
• To what extent has the exit strategy been well planned and successfully implemented, including potentials and directions for conituation of support?
• How effectively has project contributed to the establishment of effective partnerships and development of national capacities?

[1] http://web.undp.org/evaluation/guidance.shtml#outcome

[2] http://www.ilo.org/eval/Evaluationguidance/WCMS_168289/lang--en/index.htm

[3]https://www.entwicklung.at/fileadmin/user_upload/Dokumente/Zivilgesellschaft/GL_ExpenditureVerification_Dec2021.pdf

Duties and Responsibilities

The evaluation process is divided in five phases:

Activity 1: Preparation, mainly devoted to structuring the evaluation approach, compiling programme documentation, desk review of relevant reports, evaluation design, methodology and detailed work plan.

Estimated work input: 3 days

Location: Home- based

Responsible Party: International Consultant

Activity 2: Inception, which will involve consultations between the evaluation team and the UNDP and ILO, programme portfolio review, finalization of stakeholder mapping, inception meetings with the Project Board, review of the result logics, analysis of information relevant to the initiative, finalization of evaluation methodology and preparation and validation of inception report. The inception report shall also include justification for selection of two the countries for field visits and two project outputs for an in-depth analysis.

Estimated work input: 3 days

Location: Home-based

Responsible Party: UNDP, ILO, ADA, International consultant

Activity 3: Data collection and analysis, including in-depth desk research, in-depth review of the project documents and monitoring frameworks, interviews and/or online interviews as necessary, staff and partner survey/s, and field visits. Nonetheless, field trips will only take place should the health safety conditions permit. In case of imposed travel restrictions due to escalation of COVID-19 infection rates, the consultant will propose alternative evaluation plan and in agreement with UNDP, ILO and ADA, will be developed.

Estimated work input: 7 days

Location: Home-based and the two of the six countries/territories (Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia and Serbia) defined in the inception report. The geographical selection will be made according to the Project’s areas of intervention where designated project activities took place. A non-exclusive list of all stakeholders interacted within the scope of the said project will be provided for the Consultant’s reference to be used while s/he prepares the interviewee list.

Responsible Party: UNDP, ILO, International consultant

Activity 4: Data analysis and reporting stage, focusing on data analyzed, interpretation of findings and drafting and validation of an evaluation report. Draft of the report shall be validated by ADA, UNDP and ILO. The final report should provide overall and country specific recommendations for future work. In addition, the consultant shall prepare the Results-Assessment Form (Annex 1) which will be the introductory section of the Final Evaluation Report.

The evaluation does not envisage a thorough impact assessment therefore, any exploration of impact should focus on immediate impacts instead of large-scale transformational impacts. Likewise, questions pertaining to sustainability should take into consideration the scope and resources invested by the said project. The future-looking evaluation shall primarily seek to inform a possible future phase of interventions concerning inclusive labour markets in the Western Balkans. At the end of the final report, the evaluator is expected to present concrete recommendations which are addressed to specific stakeholders including the donor and implementing agencies. The final evaluation report will be reviewed and approved against a set of criteria which seeks to ensure premium quality for the evaluation deliverables (Annex 3). The Guidelines for Expenditure Verification of projects (co-)financed by the Austrian Development Agency (ADA) need to be considered throughout the entire evaluation process.

Estimated work input: 7 days

Location: Home based

Responsible Party: International consultant, UNDP, ILO, ADA

Activity 5: Dissemination, follow-up and use, once the evaluation is completed UNDP and ILO are responsible for the development of a Management Response, publishing of the evaluation report and the dissemination of evaluation findings.

The consultant will only be responsible for the preparation, inception, data collection and data analysis and reporting phase. Evaluation dissemination, follow up and use will be responsibility of UNDP and ILO.

Deliverables:

Deliverable 1 (week 1)

Evaluation inception report i.e. methodology, information collection roadmap, list of institutions/persons to be contacted, modalities for data collection (prior to start of in-country evaluation)

Review and approval required from UNDP, ILO, ADA.

Deliverable 2 (week 3)

Draft evaluation report in line with the UNDP corporate standard that can be found in annex 7 of the UNDP evaluation guidance. About 20 pages without annexes including a draft executive summary and the results assessment form

Review and approval required from UNDP, ILO, ADA.

Deliverable 3 (week 4)

Final draft evaluation report with comments incorporated (structure of the report annex 2)

25-30 pages without annexes), the final executive summary and the results-assessment form

Review and approval required from UNDP, ILO, ADA.

Payment Schedule:

Payment 1: 20% upon confirmation by the Certifying Officer of satisfactory delivery of Deliverable 1

Payment 2: 30% upon confirmation by the Certifying of satisfactory delivery of Deliverable 2

Payment 3: 50% upon confirmation by the Certifying of satisfactory delivery of Deliverable 3

Competencies

Corporate competencies:

• Demonstrates integrity by modelling the UN’s values and ethical standards;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability
• Treats all people fairly without favouritism;
• Fulfils all obligations to gender sensitivity and zero tolerance for sexual harassment

Functional competencies:

• Strong interpersonal skills, communication and diplomatic skills, ability to work in a team
• Openness to change and ability to receive/integrate feedback
• Ability to work under pressure and stressful situations 

Required Skills and Experience

Education:

Minimum Master’s Degree in Social or Economic Sciences.

Experience:

• Minimum 10 years of professional expertise in international development co-operation in programme/project management and impact assessment/evaluation
• Minimum 6 years of experience in conducting evaluations including around UNDP thematic areas of local development, employment, social inclusion, governance and public sector reform;
• Proven experience as lead consultant in at least two project evaluations of similar size and scope.
• Demonstrated field experience in data collection in qualitative research;
• Proven track record in preparing and using interview guides, interview reports,
• synthesizing reports based on fieldwork;
• Excellent professional knowledge of the ECIS region, especially Western Balkans, related to economic development, labour market reforms and social inclusion.

Language skills:

Excellent writing, editing, and oral communication skills in English

Knowledge of languages spoken in the Western Balkans shall be considered an asset

5. Evaluation of Applicants

Individual consultants will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal.

The award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:

a) responsive/compliant/acceptable, and

b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

Technical Criteria - 70% of total evaluation

Financial Criteria - 30% of total evaluation

Technical Criteria - 70% of total evaluation – max. 70 points:

• Criteria A (Education) – max points: 10
• Criteria B (Experience with international development co-operation in programme/project management and impact assessment/evaluation) - max points: 10
• Criteria C (Experience in evaluation around the project thematic areas of employment and labour market reforms) – max points: 10
• Criteria D (Experience in the region) – max points: 5
• Criteria E (Proposed methodology) - max points: 10
• Criteria F (Interview- English language, capacities in conducting evaluations, communication skills)- max points: 25

Financial Criteria - 30% of total evaluation – max. 30 points

Only candidates who will receive min. 70% of points in desk review (criteria A-E) will be invited to interview. Only candidates who will get min. 70% of points in technical evaluation (criteria A-F) will be considered for financial evaluation.

6. Application procedures

Templates can be found under https://procurement-notices-admin.undp.org/view_notice.cfm?notice_id=88824

Qualified candidates are requested to apply online via this website. The application should contain:

- Cover letter explaining why you are the most suitable candidate for the advertised position. Please paste the letter into the "Resume and Motivation" section of the electronic application.

- A brief methodology on how you will approach and conduct the work

- Filled P11 form or CV including past experience in similar projects and contact details of referees (blank form can be downloaded from: http://www.eurasia.undp.org/content/dam/rbec/docs/P11_modified_for_SCs_and_ICs.doc)

- Offeror’s Letter (blank template can be downloaded from https://bit.ly/2KO1okS).

- Financial Proposal* in USD - specifying a) total lump sum amount for the professional fee for tasks specified in this announcement. The travel costs for the missions to Albania, Bosnia and Herzegovina, Kosovo, Montenegro, Kosovo and Serbia and living allowances for 2 days in each country, territory should be presented separately. In general, UNDP does not accept travel costs exceeding those of an economy class ticket. Per diems cannot exceed maximum UN daily allowance rates (http://icsc.un.org/rootindex.asp). In country travel will be provided by UNDP Country Offices.

- Incomplete applications will not be considered. Please make sure you have provided all requested materials

https://procurement-notices-admin.undp.org/view_notice.cfm?notice_id=88824

*Please note that the financial proposal is all-inclusive and shall take into account various expenses incurred by the consultant/contractor during the contract period (e.g. fee, health insurance, vaccination, personal security needs and any other relevant expenses related to the performance of services...).

Payments will be made only upon confirmation of UNDP on delivering on the contract obligations in a satisfactory manner.

Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director. Consultants are also required to comply with the UN security directives set forth under dss.un.org

General Terms and conditions as well as other related documents can be found under: http://on.undp.org/t7fJs.

Qualified women and members of minorities are encouraged to apply.