Historique
The United Nations Development Programme (UNDP) in Sierra Leone in collaboration with development partners and civil society organizations, supported the Government of Sierra Leone (GoSL) and relevant institutions to conduct elections in 2018.? The support was channelled through the “Conflict Prevention and Mitigation (CPM) during the electoral cycle” project. The applicable Project Document (ProDoc) was signed by the Ministry of Internal Affairs (MIA) in April 2017. The project operationally started June 2017 and ended March 2019.
In accordance with UNDP and donor Monitoring and Evaluation (M&E) policies and procedures, the project is required to undergo a Terminal Evaluation (TE) upon completion of implementation. These terms of reference (TOR) set out the expectations for a TE of the project.
UNDP wishes to engage the services of an International Consultant (IC) to conduct a Terminal Evaluation of the project. The IC will be required to assess all activities undertaken within the framework of the project including comparing planned to actual outputs and assessing the actual results to determine their contribution to the attainment of the project objectives. The consultant will also attempt to evaluate the efficiency of project management, including the delivery of outputs and activities in terms of quality, quantity, timeliness and cost efficiency as well as features related to the process involved in achieving those outputs and the impacts of the project. The evaluation will also address the underlying causes and issues contribution to targets not adequately achieved.
Project description
The details of the project to be evaluated are as follows:
Project Summary:
Project title: Conflict prevention and mitigation during electoral cycle in Sierra Leone.
PBF project No: 00105794, Donor (at endorsement): Peace Building Fund (PBF): US$ 2,764,398 (June 2017 - March 2019)
DFID project No: 203878-106, Co-financing: Canada: US$ 401,517 (Jan - Mar.2018);
Canada Agreement No: 7383741, Co-financing: DFID: US$ 1,872,675 (Sept. 2017 - June 2018) Total Budget: US$ 5,038,590
UNDP Atlas Award ID: 00077436/ UNDP Atlas Project ID:00105765, Pro Doc signature (start date): 01 June 2017
Participating UN Organizations: UNDP and OHCHR- Closing Date (operational): Initial:? 31 December 2018
Current: 31 March 2019 (including 3 months No Cost Extension) Total duration: 18 Months
Lead national counterpart: Ministry of Internal Affairs- Closing Date (operational): Initial:? 31 December 2018
Current: 31 March 2019 (including 3 months No Cost Extension) Total duration: 18 Months
Implementation Partners:? Office of the Vice President, Ministry of Political and Public Affairs (MPPA), Office of National Security (ONS), Judiciary, Sierra Leone Police (SLP), Political Parties Registration Committee (PRRC), Legal Aid Board (LAB), Human Rights Commission (HRC-SL), National Commission for Democracy (NCD), BBC Media Action, Media Reform Coordination Group (MRCG), Women’s Forum, West African Network for Peacebuilding (WANEP), Campaign for Good Governance (CGG), Fambul Tok, National Election Watch (NEW), Institute for Governance Reform (IGR), and Kono District Youth Council (KDYC).
The Project involves the implementation of a range of interconnected activities across several target institutions, including Ministries, Departments and Agencies (MDAs) and Civil Society Organizations.
The project covered three phases:
Phase 1: The induction phase which focused on establishing the management structure, providing support to the training of trainers, re-activating and equipping the situation rooms and the recruitment of staff and consultants.
Phase 2: The implementation phase which focused on the full rollout of the project.
Phase 3: The concluding stages which focused on post-election activities, mid-term and terminal evaluations, audit and lessons learned.
Project objectives and outcomes
The overall objective of the project was to: support the building of a peaceful and secure environment during the 2018 electoral process through preventive and mitigating measures. The project aimed at contributing towards realization of to two outcomes: Outcome one (1): enhanced political dialogue, peace advocacy and violence prevention throughout the electoral cycle; and outcome two (2): promotion of public security, civil protection, human rights and strong national and local capacities for resolving disputes and building peace.
Devoirs et responsabilités
Evaluation scope and objectives
The evaluation presents an excellent opportunity to assess PBF’s achievements in an inclusive way and its overall added value to peacebuilding in Sierra Leone in the areas of electoral support, peacebuilding, conflict mitigation and social cohesion. The evaluation will not only help to better understand how the PBF project has progressed against its intended results, but also help inform future potential contributions of the UN Peacebuilding Fund to Sierra Leone.
Three main elements to be evaluated are Delivery, Implementation and Finances. Each component will be evaluated using the criteria: relevance, effectiveness, efficiency and sustainability.
Implementation approach
- Review the project efficiency, including its implementation strategy, institutional arrangements as well as its management and operational systems and value for money;
- Review the clarity of roles and responsibilities of the various individuals, agencies and institutions and the level of coordination between relevant players, including between UN Projects. Assess the level to which the AWP and performance indicators were used as project management tools;
- Evaluate any partnership arrangements established for implementation of the project with relevant stakeholders involved in the country /region;
- Describe and assess the efforts of UNDP and other stakeholders in support of the implementation partners, regional and national institutions;
- Make recommendations as to how to improve future projects’ performance in terms of effectiveness and efficiency in achieving impact on institutional and capacity development and the targeted concerns.
- Make assessment of the project’s management of risk.
Stakeholder participation and benefits accrued
- Assess the extent to which the representatives of the country (including MDAs, civil society, local communities etc.) were actively involved in project implementation and comment as to whether the scope of their involvement has been appropriate given the broader goals and objectives of the project;
- Review and evaluate the extent to which project benefits have reached and contributed to peaceful elections.
- Asses how inclusive the benefits of the programme were – both in terms of reaching marginalised groups, as well as in terms of geographic spread
Relevance and timeliness
Assess the relevance and appropriateness of the project in terms of: addressing the most relevant peacebuilding issues, identifying and undertaking the right interventions during the electoral process, alignment with the priorities of the Government of Sierra Leone, the project’s ability in supporting sustaining peace priorities and addressing cross-cutting issues such as gender and human rights in Sierra Leone.
Sustainability
- Assess to what extent phase - II of the project (henceforth referred to as “the PBF Project”) has made a concrete contribution in terms of building and consolidating peace in Sierra Leone and to the SDGs particularly SDG 16 & 5 and provide clear supporting evidence;
- Assess the likelihood of continuation of project outcomes/benefits after completion of the conflict prevention and mitigation funding; and describe the key factors that will require attention to improve prospects for sustainability of project outcomes. Factors of sustainability that should be considered include: institutional capacity (systems, structures, staff, expertise, etc.) social sustainability, and policy and regulatory frameworks that further the project objectives, financial sustainability.
- Document good practices, innovations and lesson learnt. Provide concrete and actionable recommendations for future programming.
Replication Approach
- Describe main lessons that have emerged in terms of strengthening: ownership; stakeholder participation; capacity building; application of adaptive management strategies; the role of M&E in project implementation. In describing all lessons learned emphasis should be made on those lessons applicable to this project.
- Make recommendations on how the lessons and experience can be incorporated into similar initiatives in the future as well as shared and disseminated.
Financial Planning
- Assess the financial control systems, including reporting and planning, that allowed the project management to make informed decisions regarding the budget;
- Assess the extent to which the flow of funds had been proper and timely from UNDP;
- Evaluate the extent of due diligence in the management of funds and financial expenditures.
Cost effectiveness
Assess the extent to which the project has completed the planned activities and met or exceeded the expected outcomes according to schedule and as cost effectively as initially planned.
Monitoring and Evaluation
Assess utilization of project’s results based-monitoring systems and implementation of monitoring and evaluation plans including any adaptation to changing conditions (adaptive management) – and specifically, assess whether the lessons, insights and recommendations of the mid-term evaluation were applied successfully to re-direct the project.
Evaluation approach and method
The evaluator is expected to frame the evaluation effort using the relevance, effectiveness, efficiency, and sustainability criteria. The evaluation will consider the overall performance of the PBF Project’s support considering the project’s result framework and other strategic priorities spelled out in project document. The broad questions to be answered are based on the OECD DAC evaluation criteria and the UN Evaluation Group standards (including those on gender mainstreaming), which have been adapted to the context at hand as follows:
Relevance and Appropriateness:
- Was the project relevant, appropriate and strategic to the main peacebuilding goals and challenges in the country at the time of the PBF Project’s implementation?
- Was the project relevant to UN’s Peacebuilding mandate and UN SDGs, particularly SDG 16 & 5?
- To what extent are the interventions relevant to the needs and priorities of the target groups/beneficiaries?
- How relevant & responsive has the PBF project been to supporting peacebuilding priorities in Sierra Leones?
- What was the relevance of the proposed ‘theory of change’ for the PBF Project?
- To what extent did the PBF project respond to peacebuilding gaps?
- To what extent did the PBF project help address women’s involvement in peace building & promotion of social cohesiveness and decision-making processes to strengthened peace building and social cohesiveness in Sierra Leone.
Efficiency:
- To what extent did PBFs’ project support achieve the results in its proposed timeline?
- How efficient was the overall staffing, planning and coordination within the project (including between the two implementing agencies and with stakeholders? Have project funds and activities been delivered in a timely manner?
- How efficient and successful was the project’s implementation approach, including procurement and other activities?
- How efficiently did the project use the steering committee?
- How well did the project collect and use data to monitor results? How well did it communicate with stakeholders and project beneficiaries on its progress? Did it use data to inform its implementation strategy?
- How well did the project communicate on its implementation and results?
- Overall, did the PBF project provide value for money? Have resources been used efficiently?
- To what extent the PBF (Phase-II) project ensured synergies within different programmes of UN agencies and other implementing organizations and donor with the same portfolio?
- Did the PBF Project make attempts and manage to ensure catalytic results, including unblocking important processes through its interventions and bringing in funding and support from other sources to its activities, areas of support and beneficiaries?
Effectiveness:
- To what extent did the PBF Project achieve its intended outcomes and contribution to strategic vision?
- To what extend did the PBF Project mainstream a gender dimension and support gender-responsive peacebuilding?
- How effective and clear was the PBF Project’s targeting strategy in terms of geographic and beneficiary targeting?
- To what extent did the PBF Project complement work with different entities, especially with UNDP and UN WOMEN, and have a strategic coherence of approach?
- How have stakeholders have been involved in the programme’s design and implementation?
- How was the program monitored and reviewed?
Impact/Sustainability/Ownership:
- To what extent did the PBF Project contribute to the broader strategic outcomes identified in the country level strategic plans and policies??
- Did the intervention design include an appropriate sustainability and exit strategy (including promoting national/local ownership, use of national capacity etc.) to support positive changes in peacebuilding in Sierra Leone after the end of the project?
- How strong is the commitment of the Government and other stakeholders to sustaining the results of PBF support and continuing initiatives, especially women’s participation in decision making processes, supported under PBF Project?
- How has the project enhanced and contributed to the development of national capacity in order to ensure suitability of efforts and benefits?
- A set of questions fine-tuned to the context should be drafted and submitted with the inception report for input and approval. The evaluator is expected to amend, complete it as part of an evaluation inception report, and shall include it as an annex to the final report.
Methodology:
The evaluation will be summative and will employ a participatory approach whereby discussions with and surveys of key stakeholders provide/ verify the substance of the findings. Proposals submitted by prospective consultants should outline a strong mixed method approach to data collection and analysis, clearly noting how various forms of evidence will be employed vis-à-vis each other to triangulate gathered information.?
Proposals should be clear on the specific role each of the various methodological approaches plays in helping to address each of the evaluation questions. The methodologies for data collection may include but not necessarily be limited to:?
Rigorous desk review of documentation supplied by country PBF team (UNDP & UN WOMEN) including: Project documents, evaluation of PBF (Phase-I), project reports, key intervention reports and policies, minutes of project board meetings and Government Peace Building meetings etc.
Key informant interviews and focus group discussions, as appropriate, with major stakeholders including country PBF team, officials from key ministries, representatives of Civil Society Organizations, Community Leaders (females & males) etc.
Survey of key stakeholders, if relevant.?
Evaluation products (key deliverables)
The evaluator will be accountable for delivery of the following key outputs and products:
Evaluation inception report: This should be prepared before field mission and should detail evaluators’ understanding of the evaluation process. The report should include a proposed schedule of tasks, activities and deliverables, designated responsibilities for each task/product. ?It should also reflect all substantive and logistical issues to be addressed to ensure the success of the evaluation. The consultant will provide the workplan and schedule to be followed throughout the assignment.
Data Collection
Data collection and field visits as per agreed methodology.
Presentation of preliminary findings.
Draft evaluation report: The project team, Evaluation Reference Group (ERG) members and key stakeholders should review and provide input to the draft report to ensure that it meets the required quality criteria and standards.
Presentation of the findings: The key findings of the evaluation should be presented to relevant stakeholders in a joint meeting to obtain participatory comments from them. A brief progress report should be submitted during consultancy detailing: activities and tasks completed to date, any challenges faced, any adjustments made in response to challenges, any deviations from timeline and explanations for deviations and any other risks and issues
Final evaluation report: A standalone document of approximately 30-40 pages (excluding any annexes/attachments) that substantiates its recommendations and conclusions. The report will include mainly the following:
A detailed record of consultations with stakeholders to be provided as part of the information gathered by the evaluator, as an annex to the main report,
If there are any significant discrepancies between the impressions and findings of the evaluation team and stakeholders these should be explained in an Annex to be attached to the final report,
A Monitoring Effectiveness Tracking Tool (METT) with evaluators comments.
The evaluation report must provide evidence-based information that is credible, reliable and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, donor focal points, UNDP Country Office, project team and other key stakeholders including Ministries, Departments and Agencies (MDAs) and project Implementation Partners (IPs). The evaluator is expected to conduct a field mission to the provinces in Sierra Leone to meet and interact with beneficiaries and field-based actors.
Management arrangements for the evaluation.
The projects’ Chief Technical Specialist (CTS) is the evaluation manager who will manage day-to-day responsibilities of the evaluation and play the central role of connecting the other key players. An evaluation reference group will be constituted comprising of key stakeholders and will work closely with the evaluation manager to guide the evaluation process and provide guidance throughout.
Compétences
- Good listening skills
- Strong drafting skills
- Ability to discuss sensitive topics at any level, from village to provincial to ministerial stakeholders using various public platforms
- Familiarity with current government policy and strategy in terms of reparations
- Ability to work as part of a multi-sectoral, cross-cultural team
- A sound comprehension of conflict and gender sensitivities
- Excellent communication skills (both written and oral), with fluency in English
- Focuses on impact and results for the client and responds positively to feedback
Qualifications et expériences requises
Education
- A Master’s degree in Social Sciences, Law, Public Policy, Political Science, Peace Studies or related field. A combination of a Bachelors’ degree with an additional 8 years of relevant work experience may be accepted in lieu of the Masters’ degree.
Professional Experience:
- Proven record of conducting similar assessments
- At least 8 years’ experience in peace building and conflict resolution activities.
- Experience dealing with UN agencies, non-profits or advocacy campaigns
- Demonstrated ability to work independently and adhere to deadlines.
The Consultant must have demonstrated expertise and experience in advanced techniques of conducting evaluation and quantitative & qualitative research. More specifically, the consultants must be an established leader in social research with demonstrated experience in;
- Designing qualitative and quantitative research methods and sampling strategies, especially with respect of gender sensitive approach.
- Designing and conducting similar evaluations particularly in peace-building, gender equity promotion and youth empowerment promotion initiatives related projects with national and international organizations.????
- Statistical analysis with strong proficiency in data analysis packages such as SPSS, excel, or NVivo.?
- Conducting evaluations, social research studies and impact studies, preferably on peace-building, reconciliation, promotion of coexistence and harmony, gender promotion and youth promotion initiatives projects.
Langage
- Must be Proficient in the English Language (oral and written).
In addition to the above criteria, the terminal evaluation international consultant should be aware of and conduct the evaluation in accordance to the UNEG ethical guideline for evaluation to ensure the credibility and integrity of the evaluation process and products.
This is available here: http://www.unevaluation.org/document/download/548
Submission process
The application should contain:
- Cover letter explaining why you are the most suitable candidate for the assignment, a description of your understanding of the consultancy assignment, a summary of the comments on the TOR, and a brief methodology on the proposed approach and conduct of the required work,
- Confirmation of Interest document (template attached),
- Updated and signed P-11 along with your CV to include qualifications/competencies and relevant experience in similar projects and contact details of at least 2 professional referees who can certify your competencies, professionalism, quality of writing, presentation and overall suitability to this TOR,
- Individual consultants will be evaluated based on a combination of factors including cover letter, credentials on offer and an interview (optional) and the offer which gives the best value for money.?
- A duly filled financial proposal attached to the last page of the CV along with all other required documentation above.
Note: Consultants are responsible for ensuring they have any necessary vaccinations/inoculations when travelling. Consultants are also required to comply with the UN security directives set forth under dss.un.org.
Submission of applications
Proposals may be submitted on or before close of business (17:00hours) on Wednesday, 30 May 2018, (West Africa time) via UNDP jobs site: http://jobs.undp.org
Evaluation Criteria and Weighting
The consultant will be evaluated against a combination of technical and financial criteria. Maximum score is 100% out of a total score for technical criteria equals 70% and 30% for financial criteria.
The technical evaluation will include the following:
- Background and minimum educational qualification as defined above-10%
- Methodology and approach to the Consultancy – 10%
- Practical previous experience relevant to the TOR - 20%
- Substantial professional knowledge and experience in the field of advanced techniques of conducting evaluation and quantitative & qualitative research. More specifically, the consultant must be an established leader in social research with demonstrated experience mentioned under 4(i) – 4(v) above -30%
Timeframe for the evaluation process
Below is the recommended duration of the assignment. The detailed schedule will be finalized with the consultant prior to the assignment. The estimated duration of the assignment is up to 30 working days and the tentative schedule is as follows:
Deliverable Timing Actual date
Preparation (home-based) Recommended: 5 days TBD
Evaluation mission and draft evaluation report (field-based) Recommended: 20 days TBD
Final report (home-based) Recommended: 5 days TBD
TOTAL 30 days
Payment modalities
The schedule and percentage payments will follow the timelines of the below-mentioned deliverables:
Payment flow Milestone
First payment (20%): Following submission and approval of inception report and its acceptance by projects’ CTS;
Second payment (40%): Following submission of the draft evaluation Report and its acceptance by UNDP CTS;
Third and final payment (40%): Following submission and approval of final evaluation report by the projects’ CTS and UNDP CO.
Payments will be made only upon confirmation of project’s CTS on delivering on the contract obligations in a satisfactory manner. The review and approval of all payments will be made by the CTS.
Annex A: List of documents to be provided (not exhaustive)
- Project Document (ProDoc),
- Annual Work Plans (AWPs),
- Results and Resources Framework (RRF),
- Project progress reports,
- Minutes of meeting for the Technical Committee,
- Mid-Term evaluation report,
- UNDP Strategic plan (2018-2021),
- Country Programme Document (CPD) for Sierra Leone (2015-2018/19),
- United Nations Development Assistance Framework (UNDAF) for Sierra Leone (2015-2018/19).
ANNEX B: UNEG Code of Conduct for Evaluators/Midterm Review Consultants, the form is attached to the Terms of Reference with the ICPN in the UNDP Procurement website:?http://procurement-notices-admin.undp.org/view_notice.cfm?notice_id=55674&view=admin