Background
Purpose
The purpose of the projects phase one Final Project Evaluation is to provide an impartial view of the Project Strengthening the Role of Local Communities/Mjesne zajednice in Bosnia and Herzegovina (MZ Project), its relevance, quality performance, management and achievements. For this purpose, an National Evaluation Consultant will be engaged.
The information, findings and recommendations generated by the evaluation at this particular time will be used by the Project Board, UNDP, SDC, Sida and by the implementing partners to strengthen the remaining Project implementation and inform future programming.
Objective
Specifically, the Final Project Evaluation aims to review and assess the relevance, effectiveness, efficiency, impact, lessons learned and sustainability of the Project. In addition, the Final Project Evaluation is expected to identify a concept and recommendations for a potential next phase of the Project.
Background information
Governance system
The complex constitutional structure stemming from the Dayton Peace Agreement is highly cumbersome. The country of 3.79 million people has 13 constitutions (state, two entities, one autonomous district and 10 cantons), 14 legal systems and more than 160 ministries. This governance structure has led to political deadlock, low investment and socio-economic inequalities, and high levels of corruption. Slow legislative processes resulting from political stalemate further hamper progress. Corruption continues to plague the country. The rule of law and the judicial system remain problematic, while the public administration is still in dire need for reforms.
Economy
Bosnia and Herzegovina is an upper middle-income country. The economy is weighed down by a huge public sector that consumes nearly 50% of the GDP to sustain itself and has elbowed out productive public infrastructure spending. At the same time, 2017 had a 3% positive economic growth, expected to reach 3.2% in 2018. Despite decrease compared to 2016, the unemployment rate stands high at 20.5%, particularly high among youth. The business environment is still weak, impeding faster economic growth. Foreign Direct Investment, although record-high in 2017 (US$950 million), may be elbowed out by the unstable political situation.
Accession to the European Union
Bosnia and Herzegovina is on the EU accession pathway, with a potential candidate country status. In a strive to encourage Bosnia and Herzegovina to resolve its ethnic divisions and qualify for membership to the EU, in late 2005 the EU foreign ministers gave the go-ahead for talks on a Stabilization and Association Agreement (SAA) with the country. The SAA was signed in 2008, but its implementation was delayed by the country's failure to make constitutional amendments called for by the European Court of Human Rights and related to the restrictions for Roma and other minorities to run for political office. The SAA entered into force in June 2015, obtaining unprecedented political support by all government levels to the Reform Agenda. In 2016 Bosnia and Herzegovina submitted a formal application to join the Union and almost 2 years later in 2018 – submitted the answers to the EU Questionnaire. The implementation of the Reform Agenda is slow.
Local governance and community development
Currently, there are 2,583 local communities (mjesna zajednica (MZs)) in Bosnia and Herzegovina, defined as areas within Local Governments (LGs). They vary greatly in their size and level of activity and tend to play a wider and more important role in rural areas. The difficulties faced by LGs – such as insufficient financial resources – have direct implications for MZs, since these depend on local government for many aspects of service delivery, and all aspects of local and community government are affected by the wider political context, in which it can be very hard to get decisions made and implemented.
Mjesne zajednice (MZs) are defined by law in both entities and in Brcko District, which empowers them to play a rather informal role in participatory government, somewhere in between that of a grass-roots civil society organisation and the formal representative government of an elected municipal council. There are significant differences between the MZs in Federation of Bosnia and Herzegovina, Republika Srpska and Brcko District, stemming from their legal frameworks. Notwithstanding these differences, in all three jurisdictions, MZs have the character of a community organisation which, as well as directly meeting various local needs, aims to improve communication with government institutions and improve citizens’ participation in local government. However, the reality is that in many parts of the country, MZs are either non-existent, inactive or controlled by the local ruling party to further its own agenda.
Although significant reforms have not been undertaken so far in the domain of local community development, some progress was seen in 2017 when the Government of Republika Srpska adopted the new Local Self-Government Strategy 2017 - 2021, placing particular attention on the role of MZ, effective and efficient local governments, improved public services and dialogue between citizens and municipalities and cities. No regulatory or policy frameworks relevant for MZs have been changed or introduced in the Federation of Bosnia and Herzegovina.
Projects response to development challenges in the area of local governance and community engagement
The Project “Strengthening the Role of Local Communities (Mjesne zajednice/MZs) in Bosnia and Herzegovina” is a joint initiative of the Governments of Switzerland and Sweden implemented by United Nations Development Programme (UNDP) in Bosnia and Herzegovina. The Project strives to improve the quality of life of the citizens of Bosnia and Herzegovina, through enhancing local services and increasing democratic accountability and social inclusion. By encouraging community-led local development, the Project works to revitalise community government of MZs, as the places where government comes closest to the people and as grass-roots organisations that serve communities of around 1,000 people, with the possibility for real participation and joint action.
More specifically, the Project interventions are structured around three main thematic pillars, Project outcomes:
Outcome 1: Citizens and governments are committed to realise the jointly agreed upon new inclusive and gender-sensitive vision of MZs in Bosnia and Herzegovina, as foundation for more participatory and accountable local governance in Bosnia and Herzegovina;
Outcome 2: Pro-active, capacitated and inter-connected MZs contribute to downward responsiveness of local governments and improved service delivery;
Outcome 3: A new regulatory framework for functional MZs developed by governments enables more inclusive, gender-sensitive and accountable local governance in Bosnia and Herzegovina.
Detailed overview of the Project results and progress is available in the Project Results Framework, Annex 1 of these Terms of Reference.
In terms of the geographical scope, the Project is present in 136 MZs within 24 (out of total 145) cities and municipalities in Bosnia and Herzegovina: Banja Luka, Bijeljina, Brcko, Gracanica, Gradacac, Gradiška, Ilijaš, Jablanica, Kljuc, Kotor Varoš, Laktaši, Ljubinje, Ljubuški, Nevesinje, Olovo, Pale Praca, Petrovo, Rudo, Sanski Most, Stari Grad Sarajevo, Tešanj, Visoko, Zenica, Žepce.
Partnerships
The main UNDP partners in the Project include:
- Government of Sweden;
- Government of Switzerland;
- Ministry of Human Rights and Refugees of Bosnia and Herzegovina;
- The Ministry of Justice of the Federation of Bosnia and Herzegovina;
- The Ministry for Administration and Local Self-Government of Republika Srpska;
- The Government of Brcko District;
- Entity Associations of Municipalities and Cities;
- Local governments;
- MZs and civil society organisations.
Financial volume of the Project:
Total Project budget for the entire duration (2015-2019) is: CHF 7,944,419 (Government of Switzerland CHF 3,972,209.5 and Government of Sweden CHF 3,972,209.5).
This, first phase of the Project is in its final implementation year (ending on 30 June 2019), which is critical for the overall Project phase success from the viewpoint of setting in place concrete replicable community governance models, raising wide awareness and engagement by MZs not only within target localities but also beyond on the future potential role of MZs within the broader local governance system; setting the contours for potential legislative changes that will sustain the results achieved.
The Project is an integral part of the UNDP Social Inclusion and Democratic Governance Sector.
Duties and Responsibilities
Main evaluation criteria of the Final Project Evaluation
The Final Project Evaluation is to answer the following questions, so as to determine the Project’s relevance, performance, results, effectiveness, efficiency, impact and sustainability, including lessons learned and forward-looking recommendations. The questions are summarized below.
Relevance
- Were the Project’s objectives and outputs relevant to the needs of the country?
- How did the Project fit into the political, social and institutional country context, and what are its potentials to adequately contribute to development processes in the future?
Effectiveness
- To what extent the Project activities were implemented and intended results and outcomes achieved?
- Were the Project’s interventions to achieve the intended results, effective?
- What are the positive or negative, intended or unintended, changes brought about by the Project interventions? This may, inter alia, include an overview of the number of beneficiaries benefiting from improved service delivery, citizen participation at the level of local community, level of local government co-financing, etc.
- To what extent and through what mechanisms has the Project managed to promote inclusiveness of marginalized groups (i.e. youth, persons with disabilities, returnees, internally displaced, minorities), gender mainstreaming and women’s empowerment?
- What are the key lessons learned and best practices identified through the Project implementation, that need to be further build upon?
- Was the M&E function systematically applied and was it appropriate to measure performance and progress towards results?
- How effective was the Project's interaction with other local level programmes/projects in Bosnia and Herzegovina to trigger synergies maximizing development results?
Efficiency
- Have resources (financial, human, technical) been allocated strategically to achieve the Project results?
- Are there any weaknesses in programme/project design, management, human resource skills, and resources?
Impact
- What are the Project effects and impact in qualitative, as well as quantitative terms on overall improvement of quality of life of citizens in targeted areas?
- To what extent are key stakeholders/final beneficiaries satisfied with the Project implementation, specifically in terms of the partnership support and what are specific expectations for the potential follow-up assistance?
- Which are the main benefits - in qualitative and quantitative terms – for each of the Project beneficiaries groups?
Sustainability
- To what extent are the Project results sustainable? How could Project results be further sustainably projected and expanded, having in mind the potential future contribution of its concept to local development and citizens engagement?
- To what extent has the Project approach (intervention strategy) managed to create ownership of the key national stakeholders?
- To what extent have the capacities of relevant duty-bearers and rights-holders been strengthened? Which are, in this regard, challenges to overcome or potentials to be unlocked in the future?
- What would be directions to expand positive effects of the project’s concept in the area of social inclusion and gender equality in the future?
- Which are the main Project products, practices and approaches holding the strongest scaling up and replication potentials?
- What are priority interventions and general recommendations, which could ensure long term sustainability of the Project’s achievements and contribute to further development of local communities and stronger social inclusion in Bosnia and Herzegovina?
The consultancy will take a broad overview of the Project area by gathering perceptions, aspirations, feedback and data from relevant partners and stakeholders for objective analysis and conduct of the evaluation. The evaluation will look to underline the key factors that have either facilitated or impeded Project implementation; the appropriateness of skill sets among Project participants and the continued need for knowledge transfer and skills to maintain the momentum of activities already set in motion. To this end, the evaluation will examine the overall performance and impact of Project components.
The evaluation needs to assess the degree to which the Project initiatives have supported or promoted gender equality, a rights-based approach, and human development. In this regard, United Nations Evaluation Group’s guidance on Integrating Human Rights and Gender Equality in Evaluation should be consulted.
Methodology
Based on the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results, UNEG Norms and Stand for Evaluations (2016) and in consultations with UNDP Country Office, the evaluation will be participatory, involving relevant stakeholders.
The evaluation will be conducted by the evaluation team composed of an International Evaluation Consultant/Team Leader and National Evaluation Consultant. The International Evaluation Consultant will lead the evaluation process and decide on planning and distribution of the evaluation workload and tasks. She/he will closely collaborate with the National Evaluation Consultant who will provide support throughout the evaluation process.
The proposed evaluation methodology consists of a preliminary desk review of Project materials and deliverables and review of existing information relevant to the Project context; preparation of an Inception report followed by field visits; preparation of a draft Evaluation Report, debriefing with stakeholders; and Final Evaluation Report write-up. An integral part of the Final Project Evaluation will also be a concept note and a set of recommendations on directions of the next phase of the Project.
The International Evaluation Consultant is required to evaluate the MZ Project Document, progress, mid-term Project review report, annual and mid-year Project reports, UNDP Quality Assurance assessments, key Project deliverables and other relevant documents. The briefing kit will be prepared by the UNDP (List of relevant documents available in Annex 2).
The International Evaluation Consultant is expected to meet representatives of the main project stakeholders (the Embassy of Switzerland in Bosnia and Herzegovina, the Embassy of Sweden and the UNDP), for an initial briefing as well as for the debriefing at the end of the assignment.
Furthermore, the International Evaluation Consultant is expected to interview the Project team, partners as well as other stakeholders as needed. To assess Project performance, approach and modalities, the Consultants will meet with key Project partners, namely the Ministry for Human Rights and Refugees of Bosnia and Herzegovina, the Ministry of Justice of the Federation of Bosnia and Herzegovina, the Ministry for Administration and Local Self-Governance of Republika Srpska and Associations of Municipalities and Cities in both entities.
In addition, through 3 – 4 field visits, the International Evaluation Consultant will meet representatives from partner local governments and MZs (beneficiaries), so as to obtain critical feedback and information on the Project activities and results and assess the Project performance and its approach and modalities. Each visit will allow for meetings with 2 – 3 partners/stakeholders in the respective region. The Consultant will also meet with representatives of other agencies active in the field of citizen participation and local community development to assess their cooperation and level of synergies with the MZ, if needed. During these meetings, it would be important to record and accumulate inputs necessary not only for the Project evaluation, but also to inform potential design of a next project phase.
The International Evaluation Consultant is encouraged to propose additional data collecting and analysis methods to ensure better insight into and review of project achievements and may use additional approaches such as simple questionnaire-based data collection.
The expected duration of the assignment is up to 20 days (up to 10 days in Bosnia and Herzegovina, including field trips) in October/early November 2018.
The Final Evaluation Report will capture the feedback by key Project partners and stakeholders as well as observations and conclusions by the Consultants. The Final Evaluation Report will particularly highlight the recommendations.
*Note: Project Logical framework against which the external evaluation will be conducted will be sent upon request to the following address: registry.ba@undp.org.
Tasks
The consultancy is divided into four principle tasks, as follows:
Task 1 – Desk review
Following the initial meeting, the International Evaluation Consultant (with the support of the National Evaluation Consultant) will conduct a detailed review of all relevant Project documents produced during its implementation. Documentation includes but is not limited to: Project document; Project annual work plans; Project reports; monitoring records, conceptual and methodological papers and analyses; empirical studies, data on implementation of CSO and MZ initiatives financed from the Project grant funds. During the desk review, the International Evaluation Consultant will focus on evaluating the Project baseline, indicators and targets, as well as the relevance, quality and adequacy of Project approach versus its objectives and intended outputs.
Task 1 will not exceed 5 days.
Task 2 – Inception report
Upon review of documentation, the International Evaluation Consultant will submit an Inception report, showing how each evaluation question will be answered by proposing methods, sources of data and data collection procedures. The Inception report should elaborate an evaluation matrix (Annex 3) for the MZ Project and propose schedule of tasks, activities and evaluation deliverables. The Inception report should follow the structure proposed in the UNDP Handbook on Planning, Monitoring and Evaluating for Development Results, p. 156-157.
Task 2 will not exceed 2 days.
Task 3 – Evaluation
Upon the approval of the Inception report and the evaluation work plan by the UNDP, the International Evaluation Consultant is expected to carry out the evaluation of the Project, via direct interviews with key stakeholders and beneficiaries from partner LG’s and MZ’s. UNDP will provide support in organization of meetings and interviews, as necessary, as well as will make all logistical and transportation arrangements.
Once the interviews are completed, the International Evaluation Consultant will analyse data and information collected (qualitative and quantitative) and draft the evaluation report including main findings and a concept for a possible follow-up direction. A contextual analysis of the environment in which the Project is working and its relevance in fulfilling a role in that environment should also be included. The report shall seek to assess Project progress, efficiency and adequacy; process and level of success of partnership building and ownership over Project products and results; the quality of Project deliverables. The report should include the data, inputs and analysis, as well as success indicators used, and an overview of the effectiveness of the Project from the perspective of various stakeholders. The evaluation will also capture the efficiency of Project organisation and management. The draft report will contain the positive or negative, intended or unintended, changes brought about by the Project and identify factors which facilitated or impeded the realization of intended objectives.
The draft Final Evaluation Report (maximum 40 pages of the main body) will be submitted to the UNDP team for initial review. The minimum structure of the evaluation report (to be written in English language) is the following:
- Executive summary;
- Introduction;
- Methodological approach;
- Evaluation findings against the main evaluation criteria;
- Lessons learned
- Main conclusions and recommendations;
- Concept note outlining key priorities to be addressed in a possible project phase two;
- Annexes.
A debriefing session will be organised with representatives of UNDP and relevant stakeholders, so as to present the preliminary findings and recommendations of the Final Evaluation Report.
Task 2 will not exceed 10 days.
Task 4 – Submission of Final Project Evaluation Report
Following the briefing session, the International Evaluation Consultant will prepare a final Project evaluation report, capturing findings and recommendations on both the Project approach and performance. Suggestions and comments gathered during the briefing session will be taken into consideration. Also, any observations that may arise from the evaluation will be incorporated into the final report.
Finally, the International Evaluation Consultant will provide a concept note outlining key priorities to be addressed in the potential next phase of the project.
Task 3 will not exceed 3 days.
DELIVERABLES AND TIMELINESS
The following deliverables are expected:
Task 1: Desk review:
- Initial meeting with the Project owners organised (Skype session is also possible); Performed desk review of documentation and the International Evaluation Consultant fully aware of the Project; 15-20 October 2018 (5 days);
Task 2: Inception report:
- Inception report, illustrating consultant’s understanding of the evaluation requirements and including detailed evaluation work-plan prepared and submitted to UNDP. 22 October 2018, (2 days);
Task 3– Evaluation:
- Interviews with stakeholders and Project beneficiaries, including 3 – 4 field visits in MZs, partner local and cantonal governments conducted and qualitative, as well as quantitative information collected as main inputs for the Final Project Evaluation 22 – 26 October 2018, (5 days) ; Briefing and validation session with Project owners conducted (Skype session also possible) 26 October 2018; Prepared draft Final Project Evaluation Report - 1 November 2018, (5 days);
Task 4 – Submission of Final Project Evaluation Report:
Recommendations on the draft Final Project Evaluation Report provided by the Project owners embedded and the Final Evaluation Report submitted to UNDP - 20 November 2018, (3 days);
IMPLEMENTATION ARRANGEMENTS AND REPORTING RELATIONSHIPS
The evaluation will be conducted by the evaluation team composed of an International Evaluation Consultant (Team Leader) and National Evaluation Consultant (National). The International Evaluation Consultant will lead the evaluation process and decide on planning and distribution of the evaluation workload and tasks. She/he will closely collaborate with the National Evaluation Consultant who will provide support throughout the evaluation process.
The International Evaluation Consultant will report to the Social Inclusion and Democratic Governance Sector leader in UNDP Bosnia and Herzegovina. A UNDP evaluation focal point will provide support to the evaluation process. The Country Office senior management will take responsibility for the approval of the final evaluation report.
Competencies
Core values
- Demonstrates integrity and fairness by modelling UN values and ethical standards;
- Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.
Core competencies
- Demonstrates professional competence to meet responsibilities and post requirements and is conscientious and efficient in meeting commitments, observing deadlines and achieving results;
- Results-Orientation: Plans and produces quality results to meet established goals, generates innovative, practical solutions to challenging situations;
- Communication: Excellent communication skills, including the ability to convey complex concepts and recommendations, both orally and in writing, in a clear and persuasive style tailored to match different audiences;
- Team work: Ability to interact, establish and maintain effective working relations with a culturally diverse team;
- Client orientation: Ability to establish and maintain productive partnerships with national partners and stakeholders and pro-activeness in identifying of beneficiaries and partners’ needs, and matching them to appropriate solutions.
Required Skills and Experience
Education:
- Advanced university degree in social sciences, political sciences, public administration or related field;
Experience:
- At least 7 years of extensive project/programme evaluation expertise and experience, with evaluations in the area of local governance and local development;
- Sound knowledge of results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
- Knowledge and experience in the area of local government, public administration and local development-related projects;
- General understanding and knowledge of the political and administrative context in Bosnia and Herzegovina;
- Knowledge and experience of the UN System and the UN Reform process;
- Understanding of the development context and working experience in Bosnia and Herzegovina is an asset;
Language requirements:
- Fluency in English language.
Other:
- Excellent computer skills (MS Office applications) and ability to use information technologies as a tool and resource.
Technical Evaluation Criteria:
- Rating based on Qualifications - Weight: 30%;
- Interview - Weight: 50%;
- Draft of the evaluation plan- Weight: 20%;
Only candidates obtaining a minimum of 70 points would be considered for the Final Evaluation.
Final Evaluation
The final evaluation score will be based on Combined Scoring Method where technical evaluation will be weighted a maximum of 70% and combined with the financial offer which will be weighted a maximum of 30%.
EVALUATION
Individual consultants will be evaluated based on the following methodology:
Cumulative analysis
When using this weighted scoring method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:
a) responsive/compliant/acceptable, and
b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.
- * Technical Criteria weight-70%
- * Financial Criteria weight- 30%
Applicants are required to submit an application including:
- Completed and signed UNDP Personal History Form (downloadable from http://www.ba.undp.org/content/bosnia_and_herzegovina/en/home/operations/jobs.html);
- Financial proposal which will separately specify professional fee, indicating number of anticipated working days, travel costs (air-ticket or use of private vehicle), living allowance, and all other applicable costs.
Please scan all above mentioned documents and upload as one attachment only online through this website.
Note:
- For an assignment requiring travel, consultants of 62 years or more require full medical examination and statement of fitness to work to engage in the consultancy.
- Due to large number of potential applicants, only competitively selected candidates will be contacted for remaining steps of the service procurement process.