Background
Background (programme/project context)
Since independence, the political leadership Kyrgyzstan has been changed twice (in 2005 and 2010) and in particular the events of April and June 2010 were violent. Inequalities in accessing justice, resources (natural, financial, political, etc.) and services are often a cause of local as well as national tensions and conflicts. Following the change in government and inter-ethnic violence in 2010, the Peacebuilding Support Office (PBSO) has been funding a range of projects supporting systemic peacebuilding in Kyrgyzstan and building institutional foundations to prevent recurrence of conflicts in future in accordance with its Peacebuilding Priority Plan (PPP).
Economic instability and disenfranchisement among youth, along with related migration trends, represent some of the greatest threats to gender equality and inclusive peacebuilding in the Kyrgyz Republic today. Women, in particular, suffer from the lack of community support and assistance mechanisms as well as the risk of exploitation. Misguided ‘patriotism’ among some people and the lack of security concerning women has contributed to growing support for the restriction of women’s liberties and freedom of movement, their right to access basic services and full participation in public life. Actors currently engaged in promoting gender equality in Kyrgyzstan lack a credible evidence base for policy formulation and programming to counter this situation.
Against this background, UN Women (lead), UNFPA and IOM offices in Kyrgyzstan are implementing a joint programme on Building the Evidence Base to Facilitate Responsive Gender Policy and Programs for Equality and Lasting Peace in Kyrgyzstan – The Gender in Society Perceptions Study that identifies critical threats to gender equality and potential conflict triggers in order to establish a credible, reliable evidence base for informed, targeted policymaking and programming for equitable gender outcomes. The programme is aligned with the Peacebuilding Priority Plan and funded under PBSO Immediate Response Facility. The programme is currently in its final stage of implementation.
Description of the programme/project
Project strategy and expected results
Primary Project Outcome: Policy making and programming pursued by state institutions, the United Nations Country Team, development partners and civil society ensures gender equality promotion and inclusive peacebuilding based on evidence. By generating a substantial, reliable evidence base on the forms and scope of gender discrimination and community-level trends that pose risks to women’s empowerment and prevent women’s full and meaningful participation in conflict resolution, the GSPS creates the conditions for significantly more effective and targeted interventions by government, UN agencies and other international organizations participating in ongoing peacebuilding efforts, NGOs, and other civil society representatives. The study is to provide concrete quantitative and qualitative data on a range of gender equality indicators, and threats to peacebuilding including contextual knowledge, attitudes and practices (KAP) linked data.
The three GSPS outputs are as follows:
- Threats to gender equality and peace are identified through a KAP study for more gender responsive policy and programming;
- National institutional capacity in gender-sensitive data collection and analysis is strengthened;
- Gender-sensitive research capacity is strengthened among universities, state researcher institutions and researchers.
The project result will be a fuller understanding of what underpins the populations’ and disaggregated sub-groups’ (male/female; age; urban/rural; ethnicity; province of residence; etc.) Knowledge, Attitudes and Practices about gender relations and the role of women and girls in areas such as political and economic participation, violence against women and girls, migration, religion and the status of women vis-à-vis men at the family and community level in the perceptions, attitudes and practices of respondents. It will provide policy-makers with intelligence on how to design policy and interventions to bridge any gap between public attitudes towards gender equality and the Constitutional guarantee of gender equality and human rights.
Project beneficiaries and stakeholders
GSPS engaged a broad range of local actors, including those from government, academia, and civil society, to carry out a nationwide study encompassing both quantitative and qualitative methods to gather information on gender equality. Through the research process, the capacity of state institutions (including universities), and civil society researchers was to enhanced by means of specialized training on data collection and analysis, gender-sensitive research methods, Do-No-Harm principles, research ethics and the subsequent application of these skills in field research. The final outcomes of the GSPS will be widely distributed among a broad range of actors and findings will be integrated into national policy and programming through a series of workshops designed to link the gender-relevant data to practicable interventions.
Through creating a clear and highly public understanding of where the most compelling threats to progression towards gender equality lie, the findings of the GSPS were to simultaneously explain the modalities of these potential conflict triggers, impose on government and civil society to effectively address these problem areas, and allow for informed, evidence-based policymaking and programming. They were to enable the UNCT, including the implementing Recipient United Nations Organisations, to assess the effectiveness of their ongoing peacebuilding projects and make adjustments in future strategic planning and project design in order to directly address the threats and risks identified by the GSPS for lasting peace and gender equality. The results of the study were to inform the development of the upcoming United Nations Development Assistance Framework for the Kyrgyz Republic, as well as the country programmes for UN Women, UNFPA, IOM, UNDP, and others.
Budget and geographical scope and timeframe
The GSPS is implemented in all provinces of the Kyrgyz Republic, namely Chui, Issyk-Kul, Naryn, Osh, Batken, Jalal-Abad, Talas regions and Bishkek and Osh cities.
Comprising of 630,001 USD funding from the Peacebuilding Support Office’s (PBSO) Immediate Response Facility and the UN Women contribution of USD 70,000, the total project budget is USD 700,001 distributed between the responsible agencies as follows: UN Women USD 410,797, UNFPA USD 248,401 and IOM USD 40,803.
The implementation period of the project is April 2015 – September 2016.
Project management
UN Women is the lead agency of the joint programme, overseeing the overall project implementation. UN Women is in charge of the qualitative research component, particularly the identification, training, and supervision of the qualitative research design team; the training and supervision of qualitative research teams across the country as they collect data; the supervision of the qualitative data analysis and the development of final report. UN Women, also making use of its own funding contribution, is initiating the joint evaluation of the project.
UNFPA manages and monitors the quantitative research component, including the design and administration of the KAP survey, implemented by its’ national partner National Statistical Committee of the Kyrgyz Republic (NSC), and the quantitative data input, cleaning and analysis.
IOM possesses substantive expertise and is responsible for the training of research teams on Do-No-Harm approach, victim-sensitive interviewing techniques, confidentiality, and secure data storage and management.
Purpose and use of the evaluation
This final evaluation is initiated by UN Women Country Office in the Kyrgyz Republic as the leading agency of the joint programme Building the Evidence Base to Facilitate Responsive Gender Policy and Programs for Equality and Lasting Peace in Kyrgyzstan – Gender in Society Perception Study (GSPS). The purpose of the evaluation is to assess the achievement of results and implementation of the GSPS programme also with a focus on capturing the lessons learned, successes and any limitations of the process of conducting the study.
The evaluation will support accountability towards the donor and the host country authorities, and organizational learning from the experience by providing guidance and strategy options for effective and successful programming and implementation of similar research programmes or projects in the future. More specifically, the results will be applied to the development of the new Strategic Note of the UN Women Country Office in the Kyrgyz Republic for 2017-2021, the country programmes for UNFPA and IOM, and to inform the upcoming United Nations Development Assistance Framework for this period. The findings and recommendations of the evaluation will moreover support application and dissemination of the programme’s results among stakeholders to advocate for evidence-based, gender-responsive programming and policy-making.
Intended users of the evaluation are the staff of UN Women, UNFPA and IOM, NSC, Government counterparts, academia, CSOs, donor community, UNCT in the Kyrgyz Republic as well as other development partners present in Kyrgyzstan and the ECA region.
Objectives
The objectives of this evaluation are to:
- Assess the extent to which the programme’s objectives are consistent with and relevant to the priorities and policies of the donor and relevant to the needs of the target groups;
- Assess effectiveness and organizational efficiency in progressing towards the achievement of the project results
- Analyse the coordination between participating UN agencies;
- Assess the sustainability of the intervention in capacitating national researchers and use of data for informed policymaking and programming in advancing gender equality and inclusive peacebuilding;
- Identify strategies for replication and up-scaling of the programme’s best practices and provide actionable recommendations on the application of the results/data;
- Assess the extent of national research capacity built in gender-sensitive research and data collection;
- Identify and document lessons learned, good practices and innovations, success stories and challenges within the programme, to inform planning of similar research programmes or projects in the future;
- Asses how the intervention and its results relate and contribute to localisation of the Sustainable Development Goals and their monitoring.
Key evaluation questions:
Relevance
- What is the relevance of the project results to national policy-making in the relevant areas of peacebuilding and gender equality? To what extent is the intervention supportive of national development strategies, government priorities and the PPP?
- To what extent was the project conceptualised and designed collaboratively, with a shared vision among UN partner agencies, and in response to CRC & CEDAW concluding observations/comments, national priorities and commitments on GE/WE, new aid modalities and the UNDAF? And to what extent national partners were involved in conceptualization and design process?
- To what extent did project design processes integrate human rights principles and strategies of the HRBA?
- To what extent did the participating agencies possess the requisite comparative advantage and strengthen same in the programme’s area of work during the implementation period in comparison with other UN entities and key partners in the Kyrgyz Republic?
Effectiveness:
- To what extent have the expected results of the project until closure been achieved on both outcome and output levels?
- To what extent did the project meet the needs and expectations of the beneficiaries?
- What are the reasons for the achievement or non-achievement of the project results? Has the project achieved any unforeseen results, either positive or negative?
- To what degree has the data collection, data processing and analysis capacity of the researchers within the qualitative research component been built, given the various trainings, guidance and feedback provided over the project? How well did the intervention succeed in strengthening national research capacities within the quantitative component?
- To what extent the project affected increased collaboration, coordination, and information exchange among the participating agencies in relation to GE/WE?
Efficiency:
- Have resources (financial, human, technical support, etc.) been allocated purposefully to achieve the project outcomes?
- To what extent did the management structure of the intervention support efficiency for programme implementation and achievement of results?
- Have the outputs been delivered in a timely manner?
- Were the methodologies selected and the research tools prepared for both quantitative and qualitative components appropriate for achieving the expected outputs?
- To what extent the coordination mechanisms successfully facilitated dialogue between both components with the aim of strengthening the research methodology and validity of findings?
- How successful were the management arrangements of the joint programme, including inter-agency coordination and cooperation?
- To what extent did the programme’s M&E actions facilitate timely tracking of the progress towards its objectives and well-informed management decisions?
Sustainability:
- How effectively has the project built national ownership of the evidence collected and the established partnerships with relevant stakeholders to ensure that evidence collected is likely acted upon?
- What is the potential and range of options for making use of the data produced and influencing/informing programming and policy-making?
- How effectively did project support development of national research capacities to ensure sustainability of efforts and benefits?
- To what extent have public/private national resources and/or counterparts been mobilized to contribute to projects’ objective and produce results?
Considering the mandates to incorporate human rights and gender equality in all UN work and the UN Women Evaluation Policy, which promotes the integration of women’s rights and gender equality principles into evaluation, these dimensions will require special attention for this evaluation and will be considered under each evaluation criterion.
It is expected that the evaluation team will develop and consecutively refine an evaluation matrix, which will relate to the above questions, the areas they refer to, the criteria for evaluating them, the indicators and the means for verification as a tool for the evaluation. The final evaluation matrix will be approved in the evaluation inception report.
Scope of the evaluation
The final evaluation of the joint programme Building the Evidence Base to Facilitate Responsive Gender Policy and Programs for Equality and Lasting Peace in Kyrgyzstan – Gender in Society Perception Study (GSPS) will be conducted at the end of the project implementation and funding period and will cover the entire project duration from March 2015 when funding was received to September 2016 when the project is to be operationally closed. The evaluation is planned to be conducted from August 2016 with the final report due by November 2016.
The evaluation includes a data collection mission to Bishkek and a field visit to Osh in Kyrgyzstan.
The relevance of the research methodology and the technical robustness of its implementation, and the research tools of the qualitative component of the programme are to be peer reviewed by an independent researcher prior to the final evaluation of the programme. The report with findings and recommendations of review will be made available for the evaluation team to inform the final evaluation.
Evaluation design (process and methods)
The evaluation will be a transparent and participatory process involving relevant UN Women stakeholders and partners in Kyrgyzstan. The evaluation will be based on gender and human rights principles and adhere to the UNEG Norms and Standards and Ethical Code of Conduct and UN Women Evaluation Policy and guidelines
The evaluation is a final programme evaluation and will employ both a summative approach focusing on capturing the lessons learned during the implementation and assessing the achievement of the results at output and outcome levels as well as a formative, forward-looking approach assessing the applicability of the results. Given the short duration of the project it has been scheduled to commence at the very end of the implementation period when results can be assessed and stakeholders can have an informed view regarding them. The evaluation will employ mixed methods including quantitative and qualitative data collection methods and analytical approaches to account for complexity of gender relations and to ensure participatory and inclusive processes that are culturally appropriate.
Methods include but are not limited to:
- Desk review of relevant documents such as project and programme documents, progress reports, financial records, meeting minutes and monitoring reports, and secondary data or studies relating to the country context and situation;
- Online consultations and discussions with Un Women, UNFPA and IOM senior management, programme and project management staff;
- Semi-structured interviews, focus group discussions, surveys with direct and indirect beneficiaries, implementing partners, donor and other stakeholders;
- Field visit to and observation at selected project sites.
Data from different research sources will be triangulated to increase its validity. The proposed approach and methodology has to be considered as flexible guidelines rather than final requirements, and the evaluators will have an opportunity to make their inputs and propose changes in the evaluation design. The methodology and approach should, however, incorporate human rights and gender equality perspectives. It is expected that the Evaluation Team will further refine the approach and methodology and submit a detailed description in the inception report.
Comments provided by the evaluation management and reference groups are aimed at providing for methodological rigor, pointing out the contextual situation, factual errors, errors of interpretation, or omission of information and must be taken on by the evaluators to ensure a high-quality product. The final evaluation report should reflect the evaluator’s fair consideration of all information provided during the assignment. Any substantive disagreements between any informants or stakeholders, or between them and the evaluators will be pointed out.
Evaluation Process
The evaluation process has five phases:
Preparation: gathering and analysing programme data, conceptualizing the evaluation approach, internal consultations on the approach, preparing the TOR, establishment of the Evaluation Management Group (EMG) and the Evaluation Reference Group (ERG), stakeholders mapping and selection of evaluation team.
Inception: consultations between the evaluation team and the EMG, programme portfolio review, finalization of stakeholder mapping, inception meeting with the ERG, review of the result logics, analysis of information relevant to the initiative, finalization of evaluation methodology and preparation and validation of inception report.
Data collection and analysis: in depth desk research, in-depth review of PBF project document, PPP and other, in online interviews as necessary, staff and partner survey/s, and field visit.
Analysis and synthesis stage: analysis of data and interpretation of findings, and drafting and validation of an evaluation report and other communication products.
Dissemination and follow-up: once the evaluation is completed UN Women is responsible for leading the development of a joint management response, publishing of the evaluation report, uploading the published report on the GATE website, and the dissemination of evaluation findings.
Stakeholder participation
The evaluation will be a consultative, inclusive and participatory process. The evaluation will be managed by the UN Women Kyrgyzstan CO with the close support of the UN Women Regional Evaluation Specialist for ECA, who reports to the UN Women Independent Evaluation Office. An Evaluation Management Group (EMG) comprising representatives from each participating agency and their delegated programme staff members will be established to oversee evaluation management, make key decisions and quality assure the different deliverables throughout the evaluation process.
The management structure will also include the Evaluation Reference Group (ERG). The purpose of the ERG that will include key stakeholders from government and CSO to facilitate the participation of relevant stakeholders in the design and scope of the evaluation, raising awareness of the different information needs, quality assurance throughout the process and in disseminating the evaluation results.
Duties and Responsibilities
Timeframe and expected outputs (since this evaluation will be conducted by an international and national evaluator the distribution of the days amongst them are indicative and based on pleminirary disrtibution of responsibilities.)
Task Timeframe/due dates Estimated#of working days
Inception phase September 2016
Inception meetings 1-2 September 2016 2
Inception report 16 September 2016 2
(including two rounds of revision)
Data collection phase September-October 2016
Desk review, on-line interviews etc. Late September 2016 9
Field visit including presentation 26 September-7 October 2016 10
of preliminary findings
Analysis and reporting phase October-November 2016
Draft report (including two rounds 28 October 2016 10
of revision)
Final report 11 November 2016 2
Evaluation communication
products
Expected deliverables
The evaluation team is expected to deliver:
An inception report
The evaluation team will present a refined scope and a detailed outline of the evaluation design and methodology, evaluation questions, and criteria for the approach for in-depth desk review and field work to be conducted in the data collection phase. The report will include an evaluation matrix and detailed work plan.? A first draft report will be shared with the evaluation management group and, based upon the comments received the evaluation team will revise the draft.? The revised draft will be shared with the evaluation reference group for feedback. The evaluation team will maintain an audit trail of the comments received and provide a response on how the comments were addressed in the final inception report.
Presentation of preliminary findings: A PowerPoint presentation detailing the emerging findings of the evaluation will be shared with the EMG for feedback. The revised presentation will be delivered to the ERG for comment and validation. The evaluation team will incorporate the feedback received into the draft report.
A draft evaluation report: A first draft report will be shared with the EMG for initial feedback. The second draft report will incorporate Evaluation Management Group feedback and will be shared with the ERG for identification of factual errors, errors of omission and/or misinterpretation of information. The third draft report will incorporate this feedback and then be shared with the ERG for final validation. The evaluation team will maintain an audit trail of the comments received and provide a response on how the comments were addressed in the revised drafts.
The final evaluation report: The final report will include a concise Executive Summary in English and in Russian and annexes detailing the methodological approach and any analytical products developed during the course of the evaluation. The structure of the report will be defined in the inception report.
Evaluation communication products: A PowerPoint/Prezi presentation and a 2-pager/infographics on the final key findings, lessons learned and recommendations both in English and Russian.
Payment will be issued in three instalments upon the satisfactory submission of the deliverables cleared by the evaluation task manager to certify that the services have been satisfactorily performed: 20% upon the signing of the contract also to defray any travel and subsistence costs, 20% upon accepting the inception report and 60% upon accepting the final evaluation report and other evaluation communication products.
Evaluation team composition and requirements
The evaluation will be conducted by an evaluation team comprising an international consultant acting as Team Leader responsible for coordination during all phases of the evaluation process, ensuring the quality of outputs and application of methodology as well as timely delivery of all evaluation products in close collaboration with the evaluation task manager and the Evaluation Management Group, and a national consultant supporting in all substantive aspects of the evaluation. During the field mission, interpretation and transportation services will be provided as necessary.
National Consultant
The duties and responsibilities of the National Consultant are as follows:
- Participate in developing the inception report outlining design, approach and methodology of the evaluation and an indicative workplan of the evaluation team within the framework of this ToR;
- Carry out collection, research and analysis of relevant documentation and other data, and reporting;
- Participate in meetings with the EMG, ERG and other stakeholders to review findings, conclusions and recommendations;
- Participate in preparation and submission of draft report, final evaluation report and communication products;
- Assist the International Team Leader and International Team Member in related matters as advised by the Team Leader;
- Produce the Russian translation of the Executive Summary of the Final Evaluation Report.
Competencies
- Sensitivity and adaptability to culture, gender, religion, nationality and age;
- Strong analytical, writing and reporting abilities;
- Strong interpersonal and communication skills, ability to work collaboratively in a team to deliver expected outputs;
- Commitment to quality products and deadlines.
Required Skills and Experience
Education:
- Master’s degree in economics, social sciences, international relations, peace and conflict studies, gender studies or related area.
Experience:
- 5 years of relevant experience and involvement in the evaluations of development projects;
- Previous knowledge of the UN system;
- Experience/knowledge of gender equality and women’s empowerment at the country level;
- Demonstrated analytical and presentation skills and ability to collect data and structure information;
- Experience with research interventions/projects considered an important asset.
Language:
- Excellent knowledge of English and Russian, working knowledge of Kyrgyz;
- Knowledge of other local languages will be considered an asset.
X. Application procedure
All online applications must include
- Offeror's Letter to UN Women confirming interest and availability for the assignment;
- Fully inclusive Financial Proposal, indicating all costs relating to the delivery of outputs as per these terms of reference with a total figure in USD for the assignment;
- Fully completed UN P11 form including past experience in similar assignments including in relation to all requirements set out for the assignment. The P11 form can be downloaded at www.unwomen.org/about-us/employment?
Kindly note that the system will only allow one attachment so the aforementioned documents will need to be merged into ONE file and uploaded. Please carefully respond to the requirements of the Terms of Reference in the P11 that you submit.
Please note that the financial proposal must be all-inclusive and take into account various expenses incurred by the consultant during the contract period itemizing the following: fee rate per working day, daily subsistence allowance rate for every day in field for the purposes of the assignment, necessary local travel expenses by the most appropriate means of transportation and the most direct economy class practicable route and any other relevant expenses required for the purposes of the assignment. There must be a total amount of all expenses given. In case the total and the sub-totals differ, the higher amount will be used for assessment while it may not necessarily be contracted after clarifications. The Financial Proposal should be provided in USD; if the proposal is provided in any other currency it would be converted as per UN Official Rate of Exchange valid on the date of closure of the vacancy announcement. Payment will be made in USD to the international consultants and in KGS to the national consultant.
The Consultant shall promote a client-oriented approach consistent with UN Women rules and regulations and commits to high standards of quality, productivity and timeliness in the delivery of tasks. The Consultant will meet and apply the highest standards of integrity and impartiality.
The Consultant must be fully dedicated to the mandate and the values of UN Women, particularly to promoting Gender Equality as a strategy to reduce conflict, improve livelihoods and ensure fairness and justice; to Women Empowerment underpinning Gender Equality promotion efforts; to inter-ethnic tolerance and concord; and to respect for diversity.
Evaluation of applicants
Candidates will be evaluated using a cumulative analysis method taking into consideration the combination of the applicants' technical qualifications and experience, and their Financial Proposal. Candidates meeting the mandatory requirements in the Terms of Reference as per the P11 submitted will be longlisted and passed on for further technical evaluation. The technical evaluation of the longlisted candidates comprises a desk review of the submitted documents and an interview. Candidates obtaining a minimum of 70% (28 points of 40) in the desk review will be shortlisted and invited for an interview. Candidates obtaining a minimum of 70% (49 points of 70) from the desk review and the interview will be further considered for financial evaluation.
The contract will be awarded to the individual consultant whose offer and performance in an interview have been evaluated and determined as:
- Technically responsive/compliant/acceptable to the requirements of the ToR; and
- Having received a highest cumulative (technical evaluation and interview, and financial evaluation) score out of below defined criteria;
- Applications without the documents indicated as required are incomplete and will NOT be considered for further assessment. Only shortlisted candidates will be contacted.
Technical Criteria for National Consultant - 70% of total evaluation - max. 70 points
Criteria Mandatory requirement Points in desk review 40 Points in interview 30
Master’s degree in X yes/no yes/no
economics, social sciences,
international relations,
peace and conflict studies,
gender studies or related area
5 years of relevant experience X yes/no yes/no
and involvement in the evaluations
of development projects
Excellent knowledge of English X yes/no yes/no
and Russian, working knowledge
of Kyrgyz
Previous knowledge of the UN system Upto 10 Upto 10
Experience/knowledge of gender equality
and women’s empowerment at the country level
Demonstrated analytical and presentation Upto 10 Upto 5
skills and ability to collect data and structure
information
Experience with research interventions/projects Upto 5 Upto 5
will be considered an important asset
Knowledge of other local languages Upto5
will be considered
Financial Criteria - 30% of total evaluation - max. 30 points
The maximum number of points assigned to the financial proposal is allocated to the lowest price proposal. All other price proposals receive points in inverse proportion.
A suggested formula is as follows: p 30 (µ/z)
Where:
- p - points for the financial proposal being evaluated;
- µ - price of the lowest priced proposal;
- z - price of the proposal being evaluated.
References
Evaluation Policy of the United Nations Entity for Gender Equality and the Empowerment of Women (UNW/2012/8): www.un.org/ga/search/view_doc.asp?symbol=UNW/2012/12&Lang=E
How to Manage Gender Responsive Evaluation. Evaluation Handbook: www.unwomen.org/en/digital-library/publications/2015/4/un-women-evaluation-handbook-how-to-manage-gender-responsive-evaluation
UN Women Global Evaluation Reports Assessment and Analysis System (GERAAS): www.unwomen.org/~/media/headquarters/attachments/sections/about%20us/evaluation/evaluation-geraasmethodology-en.pdf
Standards for Evaluation in the UN System: www.uneval.org/document/detail/22
Norms for Evaluation in the UN System: www.uneval.org/document/detail/21
Integrating Human Rights and Gender Equality in Evaluation – towards UNEG Guidance: www.uneval.org/document/detail/980
UNEG Guidance Integrating Human Rights and Gender into Evaluation: www.uneval.org/document/detail/1616
UN SWAP Evaluation Performance Indicator: www.uneval.org/document/detail/1452
UNEG Quality Checklist for Evaluation Reports: www.uneval.org/document/detail/607
UNEG Ethical Guidelines: www.unevaluation.org/document/detail/102
UNEG Code of Conduct for Evaluation in the UN: www.unevaluation.org/document/detail/100