Background

Purpose

The purpose of this evaluation is to provide a summative, independent results-focused assessment of the two complementary projects, Srebrenica Regional Recovery Programme (SRRP) and Birac Region Development and Cooperation Project (BIRAC), in terms of the overall effects and impact the interventions brought about to various stakeholders in the subjected geographic area, throughout the entire implementation period (2003-2017). The evaluation is needed at this particular time as the programme is reaching its intended scale, to generate lessons learned and recommendations which are expected to identify appropriate strategies and operational approaches to replicate or inform new UNDP’s economic generation programmes across Bosnia and Herzegovina.

Objective

In a conjoint manner, the evaluation should examine the overall performance of the two projects, their inputs and activities, and how the outputs delivered over 14 years added value for the individuals and groups in local communities in one geographic area. In a substantive analysis of the effectiveness of the projects approach, the evaluation should look into cause and effect relations within the projects, identifying the extent to which the observed changes can be attributed to the evaluated interventions.

The specific objectives of the evaluation are to:

  • Inform the Government of the Netherlands, UNDP and other relevant stakeholders on the development change resulting from the interventions, measured inter alia, by the concrete impact on the rural enterprise development, job creation, income generation and more effective local governance;
  • Identify results and successful practices of the projects that can be scaled up/replicated into the rest of the country and provide actionable, forward looking recommendations to UNDP and stakeholders for refining and scaling up support;
  • Increase the stakeholders’ knowledge about the benefits and challenges encountered during the programme’s implementation;
  • Enrich future UNDP’s country and regional programming in the field of local economic development.

Background information

This evaluation will cover the implementation of two programmes: the Srebrenica Regional Recovery Programme (SRRP) and the Birac Region Development and Cooperation Project (BIRAC). Under the SRRP framework, three municipalities were targeted: Srebrenica, Bratunac and Milici. The BIRAC framework added two municipalities to the three SRRP ones: Zvornik and Vlasenica.

The Srebrenica Regional Regional Recovery programme is a development framework that covered the municipalities in Srebrenica, Bratunac and Milici. The implementation started in September 2002. In total, four phases were realised. The overall goal of this programme was to promote the socio-economic recovery of multi-ethnic communities with strengthened local government structures.

The programme was initiated to address the specific needs of Srebrenica resulting from the wartime horrors and political obstructions in the post-war period, which made the Srebrenica area one of the most socially and economically depressed regions of Bosnia and Herzegovina (BiH) and lagging behind the rest of the country. The many issues that needed to be addressed at the initiation of the programme were: devastated infrastructures, destroyed private properties, weak governing structures with under-financed public services, weak primary health care, social services and public utilities, and lack of economic opportunities.

With time and progress made in the development of the region, the programme’s focus gradually shifted from first responding to the urgent recovery needs of the region towards economic development. The timeline of the four phases is presented below:

  • SRRP Phase 1      Implemented through 2002 to 2007        

    Focus: Emergency and recovery

  • SRRP Phase 2      Implemented through 2005 to 2009        

    Focus: Improving the quality of life

  • SRRP Phase 3      Implemented through 2010 to 2013        

    Focus: Improving the quality of life, then consolidating the changes

  • SRRP Phase 4      Implemented through 2014 to 2017        

    Focus: Consolidating the changes and preparing the region for the future

The Srebrenica Regional Recovery Programme was designed as an Area-Based Development (ABD) programme. The programme’s approach was integrated and designed so that rural economic development and local governance are simultaneously means and ends, thereby strengthening each other, to sustainable human development. Rural economic development interventions aim at addressing poverty and increasing wealth in the region, while being used as incentives, through conditionality in the allocation of resources, to build consultative mechanisms, improve municipal efficiency and accountability. At the same time, local governance and the participatory mechanisms set in place are means to rural economic development in order to ensure equity, ownership, and sustainability of planned interventions. These principles were used all along the implementation of the programme’s four phases.

In financial terms, the total implemented budget under the SRRP framework reached USD 44,808,177. The information below presents the disbursement of funds by component and phases.

                                                                Phase 1                Phase 2                Phase 3                Phase 4                Total

Overall Management                          1,506,004             1,765,288             1,962,880             616,926                 5,851,097

Local Governance                              1,840,967             1,124,537             856,299                 323,282                 4,145,085

Infrastructure                                      5,209,046             3,033,084             4,797,891             3,219,926             16,259,946

Economic Development                    4,807,849             5,695,478             4,379,647             3,669,074             18,552,048

Total                                                      13,363,866           11,618,387           11,996,716           7,829,208             44,808,177

As stated above, the second framework implemented in the region around Srebrenica is the Birac Region Development and Cooperation Project. A first phase of the BIRAC project started in September 2013 and was completed in August 2016, while the second phase started in September 2016 and will end in June 2018. A sub-project was attached to the BIRAC 2 project in June 2017. This initiative, called the First Job project, intends to promote the first employment of vulnerable categories.

The project was to build on previous and ongoing support to the Birac region municipalities and to encourage inter-municipal and regional cooperation to support bottom-up partnership approach to development and foster territorial cohesion, applying the EU LEADER-like principles in community-led local development at the sub-regional level. The project aimed to contribute to socially inclusive and integrated development of the Birac region through effective mobilisation of development partnerships and resources, setting of the joint regional development agenda and empowering of community stakeholders to undertake development actions and translate regional priorities into visibly inclusive development results. The planned output of the BIRAC 1 project was:

  • By the end of 2015, enhanced integrated and socially inclusive development of the Birac region through functional development partnerships and regional development strategy gradually translated into development results.

The second phase intends to build on the achievements of the SRRP framework and the BIRAC 1 project and further strengthen local capacities for development of the region. The BIRAC 2 project is providing key technical and financial assistance to regional development actors, in order to: i) sustain the results achieved so far; ii) enable the region to become a front-runner in inclusive local economic development by designing innovative practices that could be scaled up to other localities in Bosnia and Herzegovina; and iii) prepare local development actors for exploiting new development opportunities, especially in terms of EU accession. The intended output of the BIRAC 2 project is:

  • Birac region able to maintain its path to inclusive economic growth and fully integrated in the wider national development trends.

In financial terms, the total contributions for both projects reach USD 5.251,651.

                                                                                BIRAC 1                                BIRAC 2                                TOTAL

Overall Management                                            477,558                                 502,523                                 980,081

Local Governance                                                1,043,335                             1,061,518                             2,104,853

Economic Development                                       879,864                                 1,286,853                             2,166,717

Total                                                                       2,400,757                             2,850,894                             5,251,651

The main results of both the SRRP and BIRAC frameworks are presented below.

Economic Development

  • Dynamic infrastructure for commercial production and marketing of three sub-sectors with potential of growth (dairy, fruit, sheep and vegetables) providing directly an increased and stable income to 613 families.
  • More than 675 families have increased their revenues through support by UNDP and local authorities in the diversification of their household activities (farming and non-farming).
  • More than 50% of farmers are recipient of extension services.
  • Close to 509 formal jobs created (30% of jobs created).
  • Use of Business Development Services increased.

Infrastructure

  • More than 200 kilometres of rural roads repaired and/or improved.
  • 15 village water supply systems restored, benefiting 2,500 people.
  • Water supply systems in the towns of Srebrenica and Bratunac upgraded benefiting more than 3,500 people.
  • 2,000 children enjoy better education conditions.
  • Improved power supply to 1,500 families.
  • 90 houses reconstructed, enabling the return of displaced families to their pre-war residences.

Local Governance

  • Establishment of municipal front offices;
  • Increased capacities of local partners in project’s planning and implementation, including support to strategic planning and spatial planning in all three SRRP municipalities;
  • Support in the improvement of municipal business processes and ISO certification;
  • Support in the establishment and strengthening of a Local Action Group (LAG) that includes representatives of the public, private and civil sectors;
  • Establishment of a functioning Public Private Dialogue platform to improve the communication between the public and private sectors and resolve local bottlenecks.

Duties and Responsibilities

Main evaluation questions

The Evaluation will cover the entire implementation period (2003-2017) and all interventions implemented by the projects Srebrenica Regional Recovery Programme (SRRP) and Birac Region Development and Cooperation Project (BIRAC). It will answer the following questions, so as to determine the relevance, effectiveness, efficiency, impact of the evaluated projects as well as lessons learnt and sustainability of their achievements:

  • Were the projects’ objectives and outputs relevant to the needs of the country and consistent with the partner governments policies and priorities, EU Accession Agenda, Agenda 2030 and other effective strategic frameworks?
  • To what extent the projects’ objectives addressed the real needs and interests of the targeted groups in the specific geographic area?
  • Were the projects’ actions to achieve the projects objectives effective and efficient?
  • To what extent the projects’ intended results (outputs and outcomes) have been achieved? How these outputs and outcomes contributed to higher level changes or to the achievement of UNDP Country Programme Document (CPD) and UN Development Assistance Framework (UNDAF)?
  • What are the project effects and impact in qualitative, as well as quantitative terms from a broader development and system building perspective?
  • What are the positive or negative, intended or unintended, changes the projects brought about on various stakeholders? This inter alia, may include an overview of the number of beneficiaries with long-term benefits from the interventions (such as improved service delivery or income generation streams, number of new jobs created, number of jobs maintained, external funds attracted by local governments, SMEs, farmers and small producers supported, local governments with improved leadership and administration, sectoral ministries and development agencies relevant to business and rural development etc)
  • To what extent the economic growth of the target region, measured through jobs created and improved income streams in agriculture as well as improved business environment, can be attributed to the projects?
  • To what extent the projects managed to institutionalize and anchor the public-private dialogue and local action group mechanisms to address key challenges to economic development and employment in the specific geographic area?
  • How successful were the projects’ strategies for strengthening specific capacities of national institutions, local governments, private sector, farmers and rural dwellers? How much are public and private sectors equipped to further specialize/professionalize business development services?
  • Did the projects manage to spur investments in growth-oriented industrial and agricultural activity, resulting in the increased competitiveness of the region?
  • To what extent has the projects’ approach (intervention strategy) managed to create ownership of the key national stakeholders? Which are, in this regard, challenges to overcome or potentials to be unlocked?
  • To what extent are the intended projects results sustainable? Are local capacities, including the recently established Local Action Group, set in place for sustaining achieved results and maintaining economic growth trends while ensuring a more inclusive economic development with strong involvement of social actors? Will the projects results lead to potential actions beyond the lifespan of the projects?
  • To what extent are environment, gender equality and human rights principles respected and mainstreamed within the project implementation?
  • What are the key lessons learnt during the project implementation process? What results and successful practices of the projects could be scaled up/replicated to other regions of the country?

The consultancy will take a broad overview of the projects area by gathering perceptions, aspirations, feedback and data from relevant partners and stakeholders for objective analysis and conduct of the evaluation. The evaluation will look to underline the key factors that have either facilitated or impeded project implementation. To this end, the evaluation will examine the overall performance and impact of all the project components.

Methodology

The proposed methodology consists of a comprehensive desk review of project materials and existing information relevant to the project context, followed by field visits to selected project sights and interviews with key informants, and the final report write up. In addition to individual interviews, focus groups could be organised to serve the purpose of collecting the feedback from stakeholders and end beneficiaries. An integral part of the Final Project Evaluation will be a set of recommendations on possible scale up/replication directions of the most successful projects’ results and practices.

The Consultant is required to review the Project Documents, progress, annual reports, previous evaluation reports, UNDP Quality Assurance reports, monitoring documents, financial disbursement reports, key project deliverables and other relevant available documents. The briefing kit will be prepared by the UNDP.

The Consultant is expected to meet representatives of the main stakeholders and implementing partners (i.e. the Embassy of Netherlands in BiH, UNDP, UNICEF), for an initial briefing as well as for the debriefing at the end of the assignment.

Furthermore, the Consultant is expected to interview the project team, partners as well as other stakeholders as needed. To assess project performance, approach and modalities, the Consultant will meet with key project partners, namely:

  • the BiH MHRR,
  • the RS Ministry for Administration and Local Self-Government;
  • the RS Ministry of Agriculture, Forestry and Water Management,
  • the Federation of BiH Ministry of Displaced Persons and Refugees; and
  • the RS Association of Municipalities and Cities.

In addition, through a tailored field mission of up to 12 working days, the Consultant will meet the representatives from partner local governments, SMEs, development agencies and final beneficiaries, so as to obtain critical feedback and information on the project activities and results, and assess the projects performance, approach and modalities.

During the mission, it is expected that the Consultant will meet with at least:

  • 10 representatives of the target local authorities (Srebrenica, Bratunac, Milici, Vlasenica and Zvornik);
  • 10 representatives of local institutions, such as the centres for social welfare, the employment bureaus, the utility companies, schools;
  • 10 representatives of local associations involved in the implementation of the SRRP and BIRAC projects; and
  • 5 to 10 representatives of the private sector and at least 20 beneficiaries in rural areas.

S/he will also meet with representatives of other UN agencies and international organizations active in the field of local governance and local development to assess their cooperation and level of synergies with the projects, if needed. During these meetings, it would be important to record and accumulate inputs necessary not only for the project evaluation, but also for potential projects’ scale up/replication. The Final Evaluation Report will capture the feedback by key project partners and stakeholders as well as observations and conclusions by the Consultant.

The Consultant may propose additional elements to ensure better insight into and review of the projects achievements. For the assessment of wider effects and impact of the two projects on different target groups over the period of 14 years, mixed method evaluation is preferred, with both qualitative and quantitative approach to data collection, analysis and interpretation. Specifically, counterfactual analyses may be considered in assessing the Project’s impact by answering the question “what would have happened in the absence of the Project”. Finally, triangulating the findings of different methods is needed to ensure multiple validity of the evaluation findings.

Prior to the start of the assignment, it is expected that the Consultant will propose a work plan to be approved by UNDP. As part of the work plan, the Consultant is expected to propose the specific data collecting and analytical tools and techniques which will complement standard processes in order to obtain credible data and information, which can measure the actual effects (for example, structured questionnaires to a selected group of individuals, groups or institutions). As the evaluation will reflect assessment of the long-term project effects and impact, the Evaluation Report is expected to provide a broad understanding of processes through which project objectives and impact are achieved. Parameters outlined in the ToR should therefore be taken as indicative. It is expected that elaboration of the detailed evaluation approach to be taken, methods to be utilized will precede to the adoption of a final evaluation schedule.

As a part of the overall evaluation methodology, the Consultant will assess the extent to which the Project has addressed the issues of social and gender inclusion of vulnerable groups. Data need to be disaggregated by relevant criteria of vulnerability in order to assess whether benefits and contributions were fairly distributed by the interventions being evaluated.

The expected duration of the assignment is 40 working days, with the consultancy period to take place in the period March-April 2018.

The Evaluator will provide services ensuring high quality, accuracy and a client-oriented approach consistent with UNDP’s Evaluation Policy, UNDP Handbook for Planning, Monitoring and Evaluating for Results and the UNEG’s Guidelines for Impact Evaluation in UN Agency Evaluation Systems.

TASKS

The consultancy is divided into 3 principle tasks, as follows:

Task 1 – Desk Review

Following the initial meeting, the Consultant will conduct a detailed review of all relevant project documents produced during its implementation, including the relevant documents from the first project phase. Documentation includes, but is not limited to: project documents; project annual work plans; project reports; conceptual and methodological papers and analyses; data on implementation of grant-funded projects supported by the intervention.

Upon review of documentation, the Consultant will submit a detailed work plan for the evaluation process, including: a list of interlocutors; tentative dates and locations of visits planned; interview questions and date for the briefing session. During the desk review, the Consultant will focus on evaluating the project baseline, indicators and targets, as well as the relevance, quality and adequacy of project approach against its outputs and outcomes.

The estimated level of effort for Task 1 is 7 expert days.

Task 2 – Field Visits

Following the desk review, the Consultant is expected to carry out the evaluation of the Project, via direct interviews with key stakeholders and beneficiaries from partner localities, as elaborated in detail above in the methodology section. UNDP will provide support to the Consultant in the organization of meetings and interviews as well as in transport and other logistics, as necessary. The Consultant will also organise focus groups to validate the collected data and information through individual interviews.

The estimated level of effort for Taks 2 is 12 expert days.

Task 3 –Reporting

Once the interviews are completed, the Consultant will analyse data and information collected (qualitative and quantitative, and gender-sensitive, where possible) and draft the evaluation report including main findings and a concept for a phasing out stage. A contextual analysis of the environment in which the Project is working and its relevance in fulfilling a role in that environment should also be included both in the final evaluation report and in the concept paper. Based on the set of questions given previously within the ToR, the report shall seek to assess project progress, efficiency and adequacy; process and level of success of partnership building and ownership over project products and results; the quality of project deliverables and importantly – project impacts and effects. The report should include the data, inputs and analysis, as well as success indicators used, and an overview of the effectiveness of the Project from the perspective of various stakeholders. The evaluation will also capture the efficiency of project organisation and management. The draft report will contain the positive or negative, intended or unintended, changes brought about by the project and identify factors which facilitated or impeded the realization of intended objectives. A particular attention will be paid to the sustainability of the project achievements beyond the lifespan of the project.

The draft evaluation report will be submitted to the UNDP team for initial review. The minimum structure of the evaluation report (to be written in English language) is the following:

  1. Executive summary;
  2. Introduction;
  3. Methodological approach;
  4. Evaluation findings against the main evaluation criteria;
  5. Main conclusions and recommendations;
  6. Lessons learned;
  7. Recommendations for projects replication/scalability.

A debriefing session will be organised with representatives of UNDP and relevant stakeholders, so as to present the preliminary findings and recommendations of the final evaluation. Following the debriefing session, the Consultant is expected to prepare the Final Project Evaluation Report, capturing the comments and recommendations put forward by UNDP representatives and other stakeholders. Also, any observations that may arise from the evaluation will be incorporated into the final report. The report will be considered as finalised once approved by UNDP.

The estimated level of effort for Task 3 is 11 expert days.

DELIVERABLES AND TIMELINE:

The Consultant is responsible for the following deliverables:

Task 1 – Desk Review:

  • Initial meeting with the project team organised (Skype session); Evaluation work plan submitted by the Consultant to UNDP; Performed desk review of documents and the Consultant is fully aware of the Projects - (up to 7 days);

Task 2 – Field Visits:

  • Interviews with stakeholders and project beneficiaries conducted and qualitative, as well as quantitative information collected by the Consultant as main inputs for the Final Project Evaluation; Focus groups for validation - (up to 12 days);

Task 3 – Reporting:

  • Briefing and validation session with project owners; Prepared draft Final Project Evaluation report; Embedded inputs and comments on the Draft Report; Submission of the final Evaluation Report -  (up to 11 days);

Competencies

  • Demonstrates commitment to UNDP’s mission, vision and values;
  • Demonstrates professional competence to meet responsibilities and post requirements and is conscientious and efficient in meeting commitments, observing deadlines and achieving results;
  • Plans and produces quality results to meet established goals, generates innovative, practical solutions to challenging situations;
  • Excellent communication skills, including the ability to convey complex concepts and recommendations, both orally and in writing, in a clear and persuasive style tailored to match different audiences;
  • Ability to interact, establish and maintain effective working relations with a culturally diverse team;
  • Ability to establish and maintain productive partnerships with national partners and stakeholders and pro-activeness in identifying of beneficiaries and partners’ needs, and matching them to appropriate solutions;
  • Conceptualizes and analyses problems to identify key issues, underlying problems, and how they relate;
  • Contributes creative, practical ideas and approaches to deal with challenging situations;
  • Demonstrates substantive and technical knowledge to meet responsibilities and post requirements with excellence;
  • Ability to produce accurate and well documented records conforming to the required standard;
  • Good knowledge of administrative rules and regulations in civil society sector;
  • Responds positively to critical feedback and differing points of view;
  • Ability to handle a large volume of work possibly under time constraints;
  • Consistently approaches work with energy and a positive, constructive attitude.
     

Required Skills and Experience

Education: 

  • Advanced university degree in social sciences, political sciences, public administration or related field.

Experience: 

  • At least 7 years of extensive project/programme evaluation expertise and experience; experience in evaluations in the area of local economic development and local governance as well as experience in evaluations of UNDP projects are considered to be an asset;
  • Sound knowledge of results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
  • General understanding and knowledge of the socio-economic, political and administrative context in Bosnia and Herzegovina is considered an asset;
  • Proven analytical skills and ability to conceptualize and write concisely and clearly;
  • Proven ability to undertake professional research using both quantitative and qualitative methods;
  • Proven communication skills, and ability to interact with multiple actors including government representatives, donors and other stakeholders;
  • Proven excellent reporting and writing skills.

Language requirements: 

  • Fluency in English language.

Other:

  • Excellent computer skills (MS Office) and ability to use information technologies as a tool and resource.

EVALUATION

Individual consultants will be evaluated based on the following methodology:

Cumulative analysis

When using this weighted scoring method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:
a) responsive/compliant/acceptable, and
b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

  • * Technical Criteria weight-70%
  • * Financial Criteria weight- 30%

Applicants are required to submit an application including:

  • Completed and signed UNDP Personal History Form (downloadable from http://www.ba.undp.org/content/bosnia_and_herzegovina/en/home/operations/jobs.html);
  • Financial proposal which will separately specify professional fee, indicating number of anticipated working days, travel costs (air-ticket or use of private vehicle), living allowance, and all other applicable costs.
  • Proposed Basic Evaluation Methodology as per tasks given in the Terms of reference.

Please scan all above mentioned documents and upload as one attachment only online through this website.

Note:

  • For an assignment requiring travel, consultants of 62 years or more require full medical examination and statement of fitness to work to engage in the consultancy.
  • Due to large number of potential applicants, only competitively selected candidates will be contacted for remaining steps of the service procurement process.