Background

NOTE: Signed Offeror’s Letter to UNDP Confirming Interest and Availability - http://www.ba.undp.org/content/dam/bosnia_and_herzegovina/docs/Operations/Jobs/Offerors%20Letter%20to%20UNDP%20Confirming%20Interest%20and%20Availability.docx Letter to UNDP Confirming Interest and Availability.docx - to be sent to e-mail ba.shared.hr@undp.org with Subject: Job ID 87913.

Bosnia and Herzegovina (BiH) faces different challenges building its institutions on the road to the European Union (EU). These challenges usually arise as a reflection of the complex constitutional framework and decision making. Specifically, some of the challenges are rooted in the issue of explosive remnants of war, manifested in the large amounts of obsolete/unsafe ammunition stockpiles that are located throughout the country. In 2015, the country adopted its Reform Agenda 2015-2018, which confirms the EU trajectory of the country and outlines reform areas, including the rule of law and good governance, as well as the need for legislative improvements related to weapons control.

Faced with the overstocking of ammunition and remnants of war, storage locations of the Ministry of Defence of BiH represent a potential risk for Uncontrolled Ammunition Site Explosion (UEMS) due to large quantity of obsolete ammunition with little or no storage and/service history. The current structure of the Armed Forces of Bosnia and Herzegovina requires significantly less ammunition stocks for regular operations, therefore there is additional surplus of ammunition that is considered non-prospective in terms of usage, while at the same time most of these stocks are passed their shell life and also need to be disposed of. These circumstances represent significant risks and threats to peace, stability and development in BiH as potential UEMS could cause loss of life and property and halt development opportunities in communities that are close to Ammunition storage sites (ASS).

As per the official data of the Ministry of Defence of BiH, in June 2019 the Ministry of Defence and the Armed Forces of BiH are responsible for management of approximately 13,000 tons of ammunition stored within 12 storage sites throughout the country. In some cases, stockpiles are over 40 years old. Some types are prone to losing stability over time, hence the presence of old and potentially unstable ammunition lends a possibility of unplanned explosion at munition storage sites (UEMS). One way of mitigating the risk of unplanned explosions is through the reduction of ammunition stockpiles. Currently in BiH, reduction of ammunition surpluses is being performed through the processes of industrial disassembly (reverse process of ammunition production) and open detonation. These methods have been the primary modes of disposal in BiH since 2006.

After completion of 100% inventory inspections in April 2019, it is expected that Ministry of Defence/Armed Forces of BiH will assign additional quantity of up to 1,500 tonnes of ammunition for disposal. This quantity will include the ammunition that is going to be subjected to the regular inspection process during 2019 (because ammunition LOT’s that have been inspected in 2013 and 2014, as a case of the emergency inspection and inventory, are now coming to an end of their shelf life) as well as a result of newly introduced laboratory testing of gunpowder stability in ammunition within the capacities of the Armed Forces of BiH.

Ultimate goal of ammunition disposal intervention is to reduce total quantity of ammunition stockpile of the Ministry of Defence/Armed Forces BiH to 10,000 tonnes until the end of 2019 that is also envisaged within the scope of, the EUFOR led, Master Plan for Ammunition, Weapons and Explosives (AWE Master Plan) as a prerequisite milestone for introduction and establishment of sustainable ammunition stockpile management system of the Ministry of Defence and the Armed Forces of BiH. Based on the defined structure of the Armed Forces of BiH, the formation equipment requires approximately 7,000 tonnes of ammunition and remaining 6,000 tonnes are considered as surplus ammunition that may be disposed, donated or sold in accordance with the decision of the Presidency BiH. 

While the process of the ammunition destruction took time to develop and gain momentum, over the course of the last several years, a more structured and coordinated assistance in this respect has been provided to the institutions and people of BiH. The latest positive development significantly improved the sustainability and ownership of the process, while managing to surmount the major obstacles created by the political situation. With the creation of the Master Plan on Ammunition, Weapons and Explosives (Master Plan), the establishment of the Strategic Committee for Weapons, Ammunition and Explosive Ordnance and the Coordination Board for the Control and Disposal of Ammunition and Explosive Ordnance as its implementing body, important steps have been taken in the direction of introducing planned and synergized approach of all national and international stakeholders.

About the EU STAR Project

Project title

European Union contribution to Stockpile Management Technical Support

and Ammunition Surplus Reduction (EU STAR)

Atlas ID

00090160

Corporate outcome and output

UNDP Strategic Plan 2018-2021; Outcome 3, Output 3.2.1

Country

Bosnia and Herzegovina

Region

Europe and Central Asia/Western Balkans

Date project document signed

26 May 2017

Project dates

Start

Planned end

1 June 2017

30 November 2019

Project budget

 USD 2,999,216.31 (EUR 2,793,346.66)

Project expenditure at the time of evaluation

ca USD 2,100,000.00

Funding source

EU

Implementing party

UNDP

The EU funded Project European Union Contribution to Stockpile Management Technical Support and Ammunition Surplus Reduction (EU STAR) is designed to assists the Ministry of Defence of BiH and the Armed Forces of BiH to effectively manage explosive remnants of war and armaments in BiH, thus supporting the creation of safe and secure environment for BiH citizens and contributing to country’s international obligations related to the Arms Trade Treaty (ATT).

Building on the foundations of the EU supported project Explosive Ordnance and Remnants of War Destruction project (EXPLODE) also implemented by UNDP (2013-2016), the EU STAR project seeks to provide targeted and substantial reduction of quantities of the ammunition and the overall stocks of ammunition, followed by the intense know-how transfer from the Project to the beneficiary.

The overall objective of the project is to reduce the risks to peace, stability, and development in BiH through reducing the threats of uncontrolled explosion posed by chemically unstable and highly hazardous ammunition and remnants of war decreased in BiH.

The specific objective/outcome of the project is to considerably decrease ammunition stockpiles by 3,000 tons and to reach full capacitation and know-how transfer from the Project to the Armed Forces personnel, thus developing a sustainable demilitarization capability in the country.

The Project focuses on two specific and measurable results/outputs:

  1. The safe and secure disposal of 3,000 tons of ammunition;
  2. Support the process of institutionalization of a sustainable demilitarization capability of the Armed Forces of BiH to continue the disposal of ammunition beyond the closure of the project. Detailed outline of the Project Logical Framework is available in Annex 1 

Partnership: The project closely works with the Ministry of Defence of BiH, the Armed Forces of BiH, the EU, the European Union Force in BiH (EUFOR), the Organization for Security and Co-operation in Europe (OSCE), the United States of America (USA) and the North Atlantic Treaty Organization Headquarters in BiH (NATO HQ) on coordination and implementation of activities designed to contribute to the disposal of unsafe and unstable ammunition, and complex weapon systems. Overview of key stakeholders and partners and their roles in evaluation is provided in Annex 2.

Target groups and beneficiaries: Final beneficiaries of the EU STAR project are the citizens of all communities across the country, living nearby the 12 localities where the ammunition storages are situated (detailed map will be provided) and where the ammunitions inspections are conducted jointly by the Ministry of Defence and Armed Forces BiH and EUFOR, based on which surplus ammunitions are identified and transported to the Technical Workshop for Repair and Destruction of Ammunition (TROM Doboj) for final disposal.

Main achievements: In two years of implementation, total result achieved by the EU STAR project, until May 2019 is 2,006 tonnes of ammunition disposed. Although the project arrived late in the year (2017), the EU STAR accounted for 761 tonnes as a project annual result. Throughout 2019, the project plans to contribute with 994 tonnes of unstable ammunition in order to achieve project goal – disposal of at least 3,000 tonnes of unstable ammunition stockpiles. The results achieved in ammunition stockpile reduction will help alleviate the potential risk of unplanned explosions and the damage to communities in terms of their socioeconomic development, living and environmental conditions.

Project relevance and alignment: The Project aims to directly address the strategic goals of BiH’s Small Arms and Light Weapons (SALW) Control Strategy 2016-2020, BiH’s international legal and political commitments under the General Framework Agreement for Peace in BiH (Dayton Peace Agreement (GFAP), BiH’s strategic priority reflected in the recently submitted BiH membership application to join the European Union (EU) and BiH’s obligations stemming from EU Accession process, with focus on EU Acquis and its Chapter 24 – Justice, Freedom and Security, as well as BiH commitments under the Rule of Law and Good Governance from the BiH Reform Agenda 2015 – 2018. The Project is aligned with the pertaining EU strategies, national and UNDP development frameworks and goals (United Nations Development Framework (UNDAF) 2015-2020, UNDP Country Programme Document (CPD) 2015-2019). It contributes to targets set within the Sustainable Development Goal (SDG) 16: Peace, justice and strong institutions. Overview of relevant documentation is provided in Annex 3.

Duties and Responsibilities

Evaluation purpose, objectives and scope

Purpose

The purpose of the Final Project Evaluation is to provide an impartial review of the EU STAR project in terms of its relevance, effectiveness, efficiency, impact, sustainability, management and achievements. The information, findings, lessons learned and recommendations generated by the evaluation will be used by the Project Board, UNDP, EU and by the implementing partners to strengthen the remaining project implementation and inform future programming.

Objective

The evaluation objective is to examine the overall performance of the EU STAR, its results, inputs and activities, and how the outputs delivered added value to the efforts of the Ministry of Defence and the Armed Forces of BiH in the field of ammunition control and management. In a substantive analysis of the effectiveness of the project approach and feedback from beneficiaries, the evaluation should assess cause and effect relations within the project, including identifying the extent to which the observed changes can be attributed to the EU STAR.

In addition, the evaluation will highlight strengths, weaknesses/gaps and good practices of the project and provide forward-looking actionable recommendations to the EU and UNDP for finetuning and scaling up support in the field of sustainable life-cycle management of ammunition held by the Armed Forces of BiH.

Scope

The evaluation will assess the extent to which the specific project objective/outcome and results/outputs have been achieved since the beginning of the project and likelihood for their full achievement by the end of the project in November 2019 (based on the Project Document and its results framework). The evaluation will look into all project activities and processes implemented throughout the country, paying special attention to the localities where the surplus ammunition is stored.

Specifically, the evaluation will review and make recommendation regarding the implementation of the critical project’s aspects, such as strategies, implementation mechanisms and partnerships with Ministry of Defence and the Armed Forces of BiH as well as institutionalization and know-how transfer from the project to the personnel of the Armed Forces of BiH. To the extent possible, the evaluation will assess the relevance and influence of the project activities aimed at reduction of surplus ammunition stored in the 12 active locations scattered throughout the country and subsequent increase of the level of safety and security for the citizens living nearby these locations.

Finally, the evaluation will look into the project specificities, innovations, synergies and linkages in the national and regional context, that proved critical in producing the intended results/outputs and the factors that facilitated and/or hindered the progress in achieving the results/outputs, both in terms of the external environment and risks, as well as internal, including: weaknesses in project design, management, human resource skills, and resources.

Evaluation criteria and key questions

The EU STAR evaluation is to answer the following questions, so as to determine the project’s relevance, performance, results, effectiveness, efficiency, impact and sustainability, including lessons learned and forward-looking recommendations. The evaluation questions are summarized below.

Relevance

  • Were the project’s objectives relevant to the needs of the country, having in mind its political, social and institutional context in BiH, and what are its potentials to adequately contribute to development processes in the future?
  • To what extent is the project aligned with the relevant national development priorities in BiH, the EU enlargement policy/accession agenda, UNDP strategic objectives and Sustainable Development Goal (SDG) 16 - peace, justice and strong institutions?
  • To what extent does the programme contribute to gender equality, empowerment of women and human rights of target groups?

Effectiveness

  • To what extent were the project activities implemented and intended results and the specific objective/outcome achieved? What are the main project accomplishments? Please provide outline of a measurable overview of the project results against the indicators and their target values/statements as defined in the project logframe.
  • What are the positive or negative, intended or unintended, changes brought about by the project’s interventions? This may, inter alia, include an overview of benefits the project brought to beneficiary institutions and citizens in local communities.
  • What factors have contributed to achieving or not achieving the intended specific objective/outcome and outputs/results?
  • To what extent has the project contributed to strengthening partnership between the Ministry of Defence of BiH, the Armed Forces of BiH, international community and local communities?
  • To what extent has the project managed to institutionalize and perform the know-how transfer from the project and its contractors to the Armed Forces personnel?
  • How effective was the project’s interaction with other relevant projects, specifically other similar EU-funded initiatives in order to trigger synergies maximizing development results?

Efficiency

  • Have resources (financial, human, technical) been allocated strategically to achieve the project results?
  • Are there any weaknesses in project design, management, human resource skills, and resources?

Impact

  • What are the project effects and impact in terms of implemented project activities, both in qualitative, and quantitative terms, on the overall improvement of ammunition control and management?
  • What are the main benefits (qualitative and quantitative) for the Ministry of Defence, the Armed Forces of BiH and citizens?
  • To what extent are key stakeholders/final beneficiaries satisfied with the project implementation, specifically in terms of the partnership support and what are specific expectations for the potential follow-up assistance?

Sustainability

  • To what extent are the project outputs/results sustainable? How could project results be further sustainably projected and expanded, having in mind the potential future needs of the Ministry of Defence of BiH and the Armed Forces BiH in terms of ammunition control and management?
  • To what extent has the project approach (intervention strategy) managed to create ownership of the key national stakeholders?
  • What would be future priority interventions to ensure long-term sustainability of the project’s achievements and contribute to further ammunition control and sustainable life-cycle management of ammunition?

The evaluation needs to assess the degree to which the project initiatives have supported or promoted gender equality, a rights-based approach, and human development. In this regard, United Nations Evaluation Group’s guidance on Integrating Human Rights and Gender Equality in Evaluation should be consulted.

Methodology

Based on the UNDP Evaluation Guidelines, UNEG Norms and Stand for Evaluations (2016) and in consultations with UNDP Country Office and the European Commission (Service for Foreign Policy Instruments (FPI)/ Instrument contributing to Stability and Peace(ICSP)) the evaluation will be participatory, involving relevant stakeholders.

The National Evaluation Consultant (the Consultant) will propose an evaluation methodology and agree on a detailed plan for the assignment as a part of the evaluation Inception Report. The proposed methodology may employ any relevant and appropriate quantitative, qualitative or combined methods to conduct the Final Project Evaluation, exploring specific gender sensitive data collecting and analytical methods and tools applicable in the concrete case. The Consultant is expected to creatively combine the standard and other evaluation tools and technics to ensure maximum reliability of data and validity of the evaluation findings.

Standard UNDP evaluation methodology would suggest the following data collecting methods:   

  • Desk review: The Consultant will conduct a detailed review of the programmatic materials and deliverables including the Project Document/Description of the Action, theory of change and results framework, monitoring and project quality assurance reports, annual workplans, consolidated progress reports etc. An indicative list of documents for desk review is provided in Annex 3.
  • “Key informant interviews: The Consultant will interview representatives of main institutional partners, European Union , UNDP, other relevant stakeholders (e.g. Strategic Committee for Ammunition, Weapons and Explosives and donors, the Ministry of Defence of BiH and the Armed Forces of BiH, HoD/EUSR and Head of Operations as well as Communication Officer; European Union Forces in Bosnia and Herzegovina - EUFOR; FPI (project officer in Vienna and programme planner in Brussels); EEAS (on mine action in general)).” For the interviews, the Consultant is expected to design evaluation questions around relevance, effectiveness, efficiency and sustainability criteria, according to different stakeholders to be interviewed. An indicative list of main stakeholders that may be considered for meetings is provided in Annex 2.
  • Meetings / focus group discussions: 5 site visits will be arranged to meet with beneficiaries and stakeholders and review the results of the project;
  • Other methodologies, as appropriate, such as surveys, case studies, statistical analysis, social network analysis, etc.[1]

As an integral part of the evaluation report and specifically under the impact criteria, the Consultant will review the project effects and impact on its target groups. In this context, the consultancy is expected to gain insights from the key national and international players in the area of ammunition control and management.

Stakeholders involvement: During the evaluation process, the Consultant is expected to meet senior representatives of the UNDP, EU, key partners and stakeholders, including the Ministry of Defence of BiH and the Armed Forces of BiH, NATO, EUFOR, OSCE, etc. Initial briefing and evaluation debriefing to obtain the critical feedback on the evaluation findings, are envisaged. To assess project performance, approach and modalities, the Consultant will also meet with key project partners and stakeholders, members of Project Board and Strategic Committee for Ammunition, Weapons and Explosives. In addition, the views of representatives of local communities will be considered to obtain critical insight and information on the project activities and results. As relevant, the Consultant will also meet with representatives of other UNDP and non-UNDP implemented initiatives in BiH active in the field of ammunition control and management. During these meetings, it would be important to record and accumulate inputs necessary not only for the project evaluation, but also to highlight recommendations and advise on potential project follow-up phase.

The expected duration of the assignment is up to 20 workdays in the period October – December 2019.

Evaluation tasks / deliverables

Following the initial briefing and a detailed desk review, the Consultant will be responsible for delivering the following products and tasks:

  • Inception Report (10-15 pages) will be presented before the evaluation starts, showing how each evaluation question will be answered by proposing methods, sources of data and data collection procedures. The Inception Report should elaborate an evaluation matrix (provided in Annex 4) for the EU STAR project and propose a schedule of tasks, activities and evaluation deliverables. The Evaluation Inception Report should follow the structure proposed in the UNDP Evaluation Guidelines, p. 22-23.
  • Evaluation and data collection mission: Upon the approval of the Inception Report and the evaluation work plan by the UNDP, the Consultant is expected to carry out the evaluation. To collect data and insights on the project, the Consultant will undertake 5 site visits in BiH and have meetings and interviews with relevant stakeholders, including the Ministry of Defence of BiH, the Armed Forces of BiH, representatives of four prospective ammunition storage sites in BiH, UNDP representatives. UNDP will provide support in organization of meetings and logistical arrangements as necessary.
  • Draft Evaluation Report: Based on the findings generated through desk review and data collection missions, the Consultant will prepare and submit the Draft Evaluation Report to the UNDP team and key stakeholders for review. Structure of the Report is outlined in Annex 5.
  • Evaluation review process (and eventual dispute settlement): Comments, questions, suggestions and requests for clarification on the evaluation draft will be submitted to the Consultant and addressed in the agreed timeframe. The Consultant should reply to the comments through the evaluation audit trail document[2]. If there is disagreement in findings, these should be documented through the evaluation audit trail, while effort should be made to come to an agreement.
  • Evaluation debriefings will be held with UNDP (contracted party that administers the project), EU representatives (EUD, FPI and EUFOR) and other key stakeholders to present main findings and recommendations either face-to-face or in a form of a Skype briefing.
  • Final Evaluation Report (maximum 50 pages of the main body) should be logically structured, contain data and evidence-based findings, conclusions, lessons and actionable recommendations, and be presented in a way that makes the information accessible and comprehensible. Finally, based on the evaluation findings and in a distinct report section, the Consultant will provide forward-looking actionable recommendations, outlining key strategic priorities to be addressed in the potential next phase of the project.[3]

Evaluation team composition and required competencies

The evaluation will be conducted by a National Evaluation Consultant. The Consultant is expected to provide an independent and substantiated review of the project achievements; consider communication and visibility; capture underperformance; review coherence and inter-connectivity among initiatives within the project; assess partnership strategy; capture feedback from beneficiaries of assistance provided by the project, in light of development results; last but not least – recommend improvements that may be undertaken to ensure quality outcome, and provide strategic forward-looking recommendations, outlining pathways for the period beyond this project phase.

Evaluation deliverables and timelines

Deliverables

# of days per task for Consultant

Tentative due date

Location

Responsible

Parties

Initial meeting with the project owners and desk review;

3

30 October 2019

Online

Consultant/Evaluation Reference Groups

Inception report including detailed evaluation work-plan;

2

06 November 2019

Online

Consultant

Evaluation and data collection mission across BiH;

5

13 November 2019 

BiH

Consultant

Debriefing session held;

1

15 November 2019

Sarajevo

Consultant/Evaluation Reference Group

Draft evaluation report;

4

25 November 2019

Online

Consultant

Evaluation review process;

1

27 November 2019

Online

Evaluation Reference Group

Two pager presentation with main messages resulting from the evaluation, including: main findings, highlighting successes, challenges, and the political value of the intervention and recommendations.

1

10 December 2019

 

 

Online

 

 

Consultant

Submission of the Final Project Evaluation Report

3

 

16 December 2019

Online

Consultant

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The Consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The Consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners. The Consultant should be free from any conflict of interest related to this evaluation.[1]  

Implementation arrangements and reporting relations

The Consultant will report to the UNDP Country Office Justice and Security Sector Leader. A UNDP Evaluation Manager will be assigned to oversee and support the overall evaluation process. In addition, an evaluation reference group will be formed to provide critical and objective inputs throughout the evaluation process to strengthen the quality of the evaluation. The Country Office Senior Management will take responsibility for the approval of the final evaluation report.

TOR annexes:

Annex 1. EU STAR Logical Framework

LOGICAL FRAMEWORK FOR THE PROJECT

OVERALL OBJECTIVE

OBJECTIVELY VERIFIABLE INDICATORS

SOURCES OF VERIFICATION

 

 

 

Reduction of threat to citizens’ security from uncontrolled explosion of chemically unstable and highly hazardous ammunition and remnants of war in Bosnia and Herzegovina.

29% reduction of war remnants in surplus ammunition stockpiles.

 

Baseline (2017): 10,360 t

Target (2019): 7,360 t.

MoD records

EUFOR records

Project records

UNDP Monitoring Tool

SPECIFIC OBJECTIVE

OBJECTIVELY VERIFIABLE INDICATORS

SOURCES OF VERIFICATION

ASSUMPTIONS

To reduce the risks to peace, stability, and development in Bosnia and Herzegovina through the considerable decrease of ammunition stockpiles in the amount of 3,000 tons of unstable ammunition as well as the full capacitation and know-how transfer from the project and its contractors to the AF BiH personnel thus developing a sustainable demilitarization capability in the country

1: Extent to which the Ministry of Defense and Armed Forces of Bosnia and Herzegovina own and run the process of disposal of unsafe ammunition, utilizing already developed technological capacities manpower established under the previous EU funded projects now operated by their own personnel.

 

Baseline (2017): Limited, still significantly relaying on external assistance and coordination

Target (2019): Ministry of Defense and Armed Forces of Bosnia and Herzegovina in full ownership of the disposal process and capable to ensure its future sustainability

 

2: Extent to which the officers and soldiers of the Armed Forces BiH are capacitated through transfer of skills and knowledge in the area of demilitarization.

Baseline (2017): Limited skills and knowledge of approximately 20 officers and soldiers of the Armed Forces BiH in the area of demilitarization (as a result of the I Project Phase);

Target (2019): Significantly increased knowledge and skills in the area of demilitarization of at least 20-Armed Forced staff located in TROM, Doboj. 20 AF staff designated exclusively for disposal in TROM.

MoD reports and disposal statistics,

EUFOR reports,

Training and Doctrine Center (TRADOC) reports,

TROM, Doboj periodical and annual disposal rate statistics,

TROM, Doboj performance reports, 

Contractors disposal reports and certificates,

Project reports.

High motivation of MoD and AF BiH to decrease ammunition stockpile

 

Keen interest of MoD and AF BiH to develop and institutionalize a sustainable demilitarization capability

RESULTS

OBJECTIVELY VERIFIABLE INDICATORS

SOURCES OF VERIFICATION

ASSUMPTIONS

Result 1: War remnants management system in place ensuring significant reduction of ammunition quantities through the safe and environmentally benign disposal.

 

 

 

 

 

 

 

 

 

 

 

 

Result 2: Ministry of Defense and Armed forces of Bosnia and Herzegovina are equipped with relevant technologies, knowledge and skills (including know-how transfer from the project and its contractor to the AF BiH personnel) for the effective disposal of ammunition

1.1: Quantitates/tones of remnants of war ammunition disposed of in a safe and environmentally benign way.

Baseline (2017):  10,360 t

Target (2019): 7,360 t.

1.2. Number of Contracts between UNDP as the implementing agency and service providers for ammunition disposal signed and implemented;

Baseline: 0

Target: 7

 

1.3. Satisfactory completion of disposal services backed-up by disposal certificate provided by services provider.

Baseline:  N/A.

Target: Each single disposal certified. 

 

2.1. Number of Ministry of Defense and Armed Forces personnel (sex disaggregated) trained and capacitated in the area of disposal planning and logistics support in order to ensure efficient and sustainable implementation of the demilitarization processes.  

Baseline (year 2017):10

Target (year 2019): 20

 

  2.2. Number of know-how transfer workshops organized for the AF BIH personnel enabling the subsequent takeover of all ammunition disposal equipment, processes and know-how from the project to the personnel in TROM, Doboj.

  Baseline (year 2017): 0 no project activities in year 2017

  Target (year 2019): 10 as per budget proposal B.L. 6.2.1.

 

2.3. Number of instances in which technical assistance was provided in order to successfully utilize and repair and maintain the targeted existing demilitarization machinery and equipment.

Baseline: 0

Target: 2

 

 

Source 1: Official records and ammunition disposal certificates of the MoD ammunition verification Commission

Source 2: Various periodic MoD and AF BiH reports, i.e. the official MoD BiH Annual Reports, Inventory Reports, etc.

Source 3: Quarterly Project Board Meetings  

Source 4: Disposal certificates provided by the disposal facility/contractor

Source 5: Records, presentations and minutes from the Strategic Committee for Weapons, Ammunition and Explosive Ordnance

Source 6: Records, presentations and minutes from the Coordination Board for the Control and Disposal of Ammunition and Explosive Ordnance

Source 7: News items, press statements and other information in the electronic media and newspapers; photo and video records

Source 8: Project reports

Source 9: Equipment transfer protocols signed with the BiH authorities

Source 10: Feedback from direct beneficiaries

Source 11: Official records from the meetings with the MoD and AF BiH, Defense Industry, international partners

Source 12: Training manuals

Source 13: EU Progress Reports

Source 14: Conclusions of the Strategic Committee for Weapons, Ammunition and Explosive Ordnance

Source 15: Conclusions of the Coordination Board for the Control and Disposal of Ammunition and Explosive Ordnance

Source 16 EUFOR reports and briefings on the implementation of the Master Plan.

Security and safety ammunition disposal standards improved and institutionalized in the MoD and AF BIH

 

Willingness to maintain and further intensify the ammunition disposal pace well after the completion of the project

Annex 2. Indicative list of main stakeholders to be interviewed in evaluation

  • MoD – Ministry of Defence of Bosnia and Herzegovina;
  • AF BiH –       Armed Forces of Bosnia and Herzegovina;
  • MoS – Ministry of Security of Bosnia and Herzegovina;
  • MOFTER - Ministry of Foreign Trade and Economic Relations o Bosnia and Herzegovina;
  • EU – European Union;
  • US Embassy – United States Embassy in Bosnia and Herzegovina;  
  • EUFOR – European Union Forces in Bosnia and Herzegovina;
  • OSCE – Organization for Security and Co-operation in Europe;
  • NATO –         North Atlantic Treaty Organization;
  • SEESAC – South Eastern and Eastern Europe Clearinghouse for Small Arms and Light Weapons;

Annex 3. List of documents to be considered for the evaluation desk review

Policies and strategies

  • EUFOR-led Master Plan on Ammunition, Weapons and Explosives;
  • Small Arms and Light Weapons (SALW) Control Strategy 2016-2020;
  • BiH Reform Agenda 2015 – 2018.

Relevant Project knowledge products, monitoring reports, studies and publications

  • UNDP Country Project Document for Bosnia and Herzegovina 2015-2019;
  • United Nations Development Assistance Framework for Bosnia and Herzegovina 2015-2020;
  • EU STAR Description of Action with annexes;
  • EU STAR Annual Progress Report (June 2017 to June 2018);

Annex 4. Required Evaluation Matrix Template

Relevant evaluation criteria

Key Questions

Specific Sub-Questions

Data Sources

Data collection Methods / Tools

Indicators/ Success Standard

Methods for Data Analysis

 

 

 

 

 

 

 

 

Annex 5. Standard outline for the UNDP evaluation report

  1. Title and opening pages with details of the project/project/outcome and of the evaluation team.
  2. Project and evaluation information details: title, Atlas number, budgets and project dates and other key information.
  3. Table of contents.
  4. List of acronyms and abbreviations.
  5. Executive summary: a stand-alone section of maximum four pages including the quality standards and assurance ratings.
  6. Introduction and overview. What is being evaluated and why?
  7. Description of the intervention being evaluated. Provides the basis for report users to understand the logic and evaluability analysis result, assess the merits of the evaluation methodology and understand the applicability of the evaluation results.
  8. Evaluation scope and objectives. The report should provide a clear explanation of the evaluation’s scope, primary objectives and main questions.
  9. Evaluation approach and methods. The evaluation report should describe in detail the selected methodological approaches, methods and analysis.
  10.  Data analysis. The report should describe the procedures used to analyse the data collected to answer the evaluation questions.
  11. Findings and conclusions. Evaluation findings should be based on an analysis of the data collected and conclusions should be drawn from these findings. Specifically, the EUSTAR Evaluation Report will include a review of impact and effects of EU STAR project on its beneficiary institutions (The Ministry of Defence of BiH and the Armed Forces of BiH).
  12. Recommendations. The report should provide a reasonable number of practical, feasible recommendations directed to the intended users of the report about what actions to take or decisions to make.
  13. Forward-looking actionable recommendation for the EU STAR project, outlining key strategic priorities to be addressed in the potential next phase of the project.
  14. Lessons learned. As appropriate and as requested in the TOR, the report should include discussion of lessons learned from the evaluation of the intervention.
  15. Annexes.

[1] UNDP Evaluation Guidelines, Box 7. Sources of conflict of interest in evaluation

[1] UNDP Evaluation Guidelines, Annex 2. Summary of common data-collection methods/sources used in UNDP evaluations

[2] Template available at http://web.undp.org/evaluation/guideline/documents/PDF/UNDP_Evaluation_Guidelines.pdf, p. 25

[3] Evaluation Report Template available at http://web.undp.org/evaluation/guideline/documents/PDF/UNDP_Evaluation_Guidelines.pdf, p.49

Competencies

Core values:

  • Demonstrates integrity and fairness by modelling UN values and ethical standards;
  • Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Core competencies:

  • Demonstrates professional competence to meet responsibilities and post requirements and is conscientious and efficient in meeting commitments, observing deadlines and achieving results;
  • Results-Orientation: Plans and produces quality results to meet established goals, generates innovative, practical solutions to challenging situations;
  • Communication: Excellent communication skills, including the ability to convey complex concepts and recommendations, both orally and in writing, in a clear and persuasive style tailored to match different audiences;
  • Team work: Ability to interact, establish and maintain effective working relations with a culturally diverse team;
  • Client orientation: Ability to establish and maintain productive partnerships with national partners and stakeholders and pro-activeness in identifying of beneficiaries and partners’ needs and matching them to appropriate solutions.

Required Skills and Experience

Academic Qualifications/Education:

  • Advanced university degree in mechanical engineering, social sciences, economics, public administration, or other sciences sustainable development; MA in any of indicated fields is considered an advantage.

Experience:

  • At least 5 years of extensive expertise and experience with evaluations in the area of defence industry, ammunition ordnance management and control; 
  • Sound knowledge of results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
  • Expertise in the area of ammunition control and management;
  • General understanding and knowledge of the political/administrative and development context of BiH;
  • Proven analytical skills and ability to conceptualize and write concisely and clearly;
  • Experience in evaluating ICSP projects is considered an advantage.

Languages Requirements:

  • Fluency in English language.

Other:

  • Excellent computer skills (MS Office applications) and ability to use information technologies as a tool and resource.

Technical criteria:

Criteria

Weight

Max. Point

Ratings based on Shortlisting Criteria

30%

30

Methodology

25%

25

Phone Interview/Interview by Skype

45%

45

Long/Shortlisting Criteria

Criteria

Points

Relevant Education

max 30 points (20 points allocated for MSc/MA; + up to 10 points for PhD).

Relevant professional experience

max 60 points

Knowledge of English

max 10 points - will be assessed as 10 points for fluency and the points decrease as per the level mentioned in the CV: good - 8 points; fair/upper intermediate – 6 points; intermediate - 4 points; beginner - 2 point.

Only candidates obtaining a minimum of 60 points would be considered for the Technical Evaluation

EVALUATION

Individual consultants will be evaluated based on the following methodology:

Cumulative analysis

When using this weighted scoring method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:
a) responsive/compliant/acceptable, and
b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

  • * Technical Criteria weight-70%
  • * Financial Criteria weight- 30%

Applicants are required to submit an application including:

  • Proposal (outlining the specific design and methods for the evaluation):
    • Explaining why they are the most suitable for the work;
    • Provide a brief methodology on how they will approach and conduct the work;
    • The methodology should present the Consultants approach, proposed detailed methods, scope and evaluation criteria and questions;
    • The methodology should apply a mixed-method approach collecting both quantitative and qualitative data to validate and triangulate data;
    • The methodology should include the filled in evaluation matrix (Annex 3);
    • The methodology should explain the data collection tool/s to be used.
  • Personal CV including relevant experience and contact details of at least 3 references;

Please scan all above mentioned documents and upload as one attachment only online through this website.

Note:

  • For an assignment requiring travel, consultants of 65 years or more require full medical examination and statement of fitness to work to engage in the consultancy.
  • Due to large number of potential applicants, only competitively selected candidates will be contacted for remaining steps of the service procurement process.