Historique

Ethiopia has manifested its commitment in the last decade to advancing the rights of women and girls and promoting gender equality by the adoption of national and international conventions such as the Convention on the Elimination of all Forms of Discrimination Against Women (CEDAW), Beijing Declaration and Platform for Action (BDPfA), Sustainable Development Goals (SDGs), and Africa Agenda 2063. The country has also adopted policy frameworks that focus on the rights of women such as the National Action Plan on Gender Equality (2006 -2010) and the Women Development and Change Strategy and Package (March 2017). [1] The Government of Ethiopia (GoE) has also put in place institutional framework to foster the implementation of laws and policies including a dedicated ministry responsible for women, children and youth that coordinates, facilitates and monitor progress and hold sectors accountable on their performance on GEWE.

The Growth and Transformation Plan (GTP) II is the overarching national framework guiding the country’s development agenda. GTP II stipulates the participation and empowerment of women as one of its strategic pillars.[2] The national policy and strategy frameworks on gender equality and women’s empowerment are strengthened by the international commitments the country has adopted. The Sustainable Development Goals (SDGs) set out 17 Goals with 169 associated targets. Goal 5 focus on the achievement of gender equality and the empowerment of all women and girls. The adoption of the SDGs in Ethiopia is precedented by the success registered in the implementation of the Millennium Development Goals (MDGs), whereby the Country achieved most of the MDGs.[3]

Ensuring the implementation of these national and global commitments on gender equality and the empowerment of women and girls requires an implementation, accountability and monitoring mechanism supported by quality data and statistics. As such, data and statistics have become an indispensable tool for devising policies to achieve gender equality and women’s empowerment, assessing their impact and ensuring accountability. Statistics, in general, and gender statistics, in particular, play a dual role to meet SDGs and the development plans in GTP II. On one hand, the availability of gender statistics promotes evidence-based decision making to augment their implementation. On the other hand, it serves as a measurement and accountability tool to monitor and evaluate the impact and effectiveness of said policies.

With the aim of responding to the data needs under the SDGs, the Entity for Gender Equality and Women’s Empowerment (UN Women), launched a global flagship programme initiative (FPI) on Making every woman and girl count: Supporting SDG monitoring and implementation through the production and use of gender statistics (MEWGC) in 2016. The programme has an overall goal of: Gender statistics is available, accessible, analysed and used to inform policymaking, advocacy and accountability for delivering gender equality and women’s empowerment. UN Women identified 12 countries (pathfinders) to develop and support the implementation of the programme between 2016 and 2021. In the meantime, other Country Offices adopted the programme as a self-starter based on the data needs in their respective countries. Ethiopia is one of these self-starter countries implementing MEWGC with the national statistical office and other relevant stakeholders.

The programme on Making Every Woman and Girl Count in Ethiopia was launched in January 2019. The programme envisions the undertaking of a midterm evaluation after one and half year of implementation of the programme. The midterm evaluation exercise has the objective of improving programme management and results framework. It will inform the strategies for future programme implementation and will serve for organizational learning and accountability.

Therefore, UN Women Ethiopia Country Office is seeking a team of consultants to conduct the midterm evaluation of the programme. The evaluation should follow the guiding documents for evaluation at UN Women, including the Evaluation Policy, Evaluation Chapter of the POM, the GERAAS evaluation report quality checklist, the United Nations System-wide Action Plan Evaluation Performance Indicator (UN-SWAP EPI) and the UN Women Evaluation Handbook on gender responsive evaluation. These documents serve as the frame of reference for the Evaluation Manager and the evaluation consultant(s) for ensuring compliance with the various requirements and assuring the quality of the evaluation report.

II. Description of the programme

The three-year Programme "Making Every Woman and Girl Count in Ethiopia: Supporting the Monitoring and Implementation of the SDGs through better Production and Use of Gender Statistics” was launched in January 2019 with an end date of December 2021[4] and ran for one and half year so far with a total estimated planned budget of 3,225,239 USD and actual budget of 1,110,580 USD as of 2019.[5] The programme is so far being implemented at the federal level and with selected CSOs that are working on SDG goal 5.

The overall goal of the programme is making gender statistics available, accessible, analysed, and used to inform policy making, advocacy and accountability for delivering on gender equality and women’s empowerment commitments.

The three-year programme has the following expected outcomes:

• Strengthened policy and financial environment is in place to enable gender-responsive national adaptation and effective monitoring of the SDGs and GTP II;
• Strengthen the production of gender statistics to enable the monitoring of national policies and reporting commitments under the SDGs; and
• Gender statistics are accessible to all users (including governments, civil society, academia, and private sector) and can be analysed to inform research, advocacy, policies, and programmes and promote accountability.

With a view to achieving the expected outcomes, the programme, has deployed the following strategies:

• Data management;
• Capacity development;
• Partnerships and cross-learning;
• Awareness creation, advocacy, dialogue forum and knowledge generation.

The programme has a three-level intervention. At policy level the legal and institutional frameworks that have an impact in the production and use of data will be reviewed, data gaps identified and enabling policy frameworks will be promoted. Secondly, the capacity building targeting the national statistical system as a whole aims at ensuring the availability, accessibility and use of quality, timely, regular, and user-friendly gender data in the country focusing on data producers. Lastly, the further analysis and dissemination of data will extend to the community level in an effort to improve the wider use of gender statistics. As the goal of the programme is to facilitate the implementation of SDGs aligned with GTP II, strengthening accountability mechanisms on the implementation of the GEWE commitments will be another focus area of the program. Multi-stakeholder coordination systems to bring accountability in the implementation of GEWE commitments will be strengthened.

The major implementing partners (IP) under this programme are Central Statistical Agency (CSA) and CSA branches, Planning and Development Commission (PDC), Ministry of Finance and Economic cooperation (MoFEC), Ministry of Women, Children and Youth (MoWCY), Bureau of Women, Children, and Youth Affairs (BoWCYA), and Civil Society Organisations (CSOs) among other.

Major donors of the programme to date include:

• Government of Sweden, through the Embassy of Sweden in Ethiopia  
• Government of Norway, through the Royal Norwegian Embassy in Ethiopia

Under the overall supervision of the UN Women Representative to Ethiopia, Africa Union and UNECA, and with the technical guidance of UN Women East and Southern Africa Regional Office (ESARO), the programme was managed by the UN Women ECO Coordination Team. The team was composed of the following staff:

• Programme manager (PM) with the overall responsibility of providing technical support and capacity-building for high quality implementation and guarantying high-quality financial management and reporting to UN Women and donors.
• Programme officers with the role of supporting the PM and strengthen the programme's intended outputs in this area while also providing support to the programme partners.
• Programme associate with the role of providing technical support to CSO partners and ensuring UN Women financial and procurement policies are strictly followed and used for intended purpose.
• Short-term technical consultants based at CSA, PDC and MoWCY to support with the implementation, monitoring and reporting of the programme.

III. Purpose (and use of the midterm evaluation)

As per the Programme Document where the need for conducting a Midterm review after a year and a half of implementation is stated, UN Women ECO is initiating a mandatory midterm evaluation of the entire MEWGC in Ethiopia programme. The purpose of the evaluation is to provide an in-depth assessment of the results against the three outcomes of the programme and performance in terms of the relevance, effectiveness, efficiency, sustainability, impact, inclusiveness, participation, equality, non-discrimination, and social transformation. It further aims to adjust the result framework of the programme through concrete recommendations to respond to the changes in the context, availability of funds and to ensure effective result measurement for the rest of the programme implementation period.

It should identify lessons learned, good practices, and factors that facilitated/hindered achievement. Through this, it aims to contribute to accountability, learning and decision-making including practical recommendations to inform the management and coordination of programme implementation leading towards the final evaluation and other related initiatives on gender statistics.

The midterm evaluation will be undertaken with the following key stakeholders in mind:

• Relevant staff from IPs, including federal government institutions
• CSOs and women organisations that have benefited from the capacity building initiatives and data and statistics produced through the program
• UN Women ECO programme staff
• UN sister agencies, such as UNDP, UNICEF and UNFPA
• Development partners

IV. Objectives (evaluation criteria and key questions)

The specific objectives of the evaluation are to:

• Assess the context under which the programme has been implemented and the extent to which the results of the programme are achieved or are on track,  including unintended results and examine to what extent the programme is aligned with relevant international agreements and conventions, national needs, government priorities as well as with the UNDAF.  
• Check on availability of data and evidence to allow informed and credible analysis of performance, and the ‘evaluability’ of the programme with a view to make amends for the implementation of the rest of the programme.
• To take stock of changes in the programming context (including normative developments, new funding available) and programmatic response.
• Review the programme design, implementation strategy, institutional arrangements as well as management and operational systems.
• To analyse and reflect on the progress of the programme and the validity of its identified strategies.
• Assess effectiveness and organizational efficiency in progressing towards the achievement of gender equality and women’s empowerment results as defined in the intervention, with a special focus on innovative and scalable and replicable interventions.
• Assess the relevance of the contribution of the programme to the national statistics development strategy in Ethiopia with particular focus on gender statistics.
• Assess the sustainability of the intervention in meeting the demand of quality, accessible, timely and reliable data to track progress on gender equality and women’s empowerment.
• Determine the impact of the intervention with respect to gender equality and women’s empowerment.
• Analyse how human rights approach and gender equality principles were integrated in implementation.
• Document good practices, innovations and lessons learnt and provide concrete and actionable recommendations for future programming.
• To provide recommendations and practical suggestions on how to revise the programme results framework particularly focusing on indicators, baseline and targets document where necessary and enhance its ability to gauge change for the rest of the program implementation period (2020-2021).

The final evaluation question and relevant evaluation instruments will be determined during the inception stage in consultation with evaluation management group and reference group. The evaluation should be based on the following criteria:

Relevance

• To what extent is the intervention relevant to the needs and priorities as defined by beneficiaries? Are they aligned to national priorities?
• To what extent is the intervention aligned with relevant national, regional and international normative frameworks for gender equality and women’s empowerment?
• What are UN Women’s comparative advantage in this area of work compared with other UN entities and key partners?
• Is the intervention logic coherent and realistic? What needs to be adjusted?
• How strategic are partners in terms of mandate, influence, capacities and commitment? To what extent had implementing partners added value to solve the development challenges stated in the programme document?
• How appropriate and useful are the indicators described in the programme document in assessing the programme's progress? Are the targeted indicator values realistic and can they be tracked? If necessary, how should they be modified to be more useful? Are the means of verification for the indicators appropriate?

Effectiveness

• What has been the progress made towards achievement of the expected outcomes and expected results? What are the results achieved so far?
• What are the internal and external factors that contributed to the achievement or non-achievement of intended results?
• To what extent did the programme contribute to achievement of results in terms of making gender statistics available, accessible, analysed, and used to inform policy making, advocacy and accountability for delivering on gender equality and women’s empowerment commitments?
• How have stakeholders been involved in the programme implementation?
• How was the programme monitored and reviewed? To what extent was this exercise useful and used? Have any good practices, success stories, lessons learned, or transferable examples been identified and documented?

Efficiency

• To what extent does the management structure of the intervention support efficiency for programme implementation?
• Have resources been used efficiently? Have activities supporting the strategy been cost-effective?
• Have programme funds and activities been delivered in a timely manner? If not, what were the bottlenecks encountered?
• Are there enough resources (financial, time, human resources) allocated to integrate human rights and gender equality in the design, implementation, monitoring and evaluation of the programme?
• Were there any constraints (e.g. political, practical, and bureaucratic) to addressing the gender data gap efficiently during implementation? What level of effort was made to overcome these challenges?

Impact

• To what extent was gender equality and women’s empowerment advanced as a result of the programme implementation?
• To what extent can the changes that have occurred as a result of the MEWGC in Ethiopia Programme be identified and measured?
• How did the programme contribute to the identified changes?
• What were the unintended effects, if any, of the intervention?
• What are the notable impacts of the programme on the lives of women and men?
• How did the interventions of this programme impact the empowerment of women at all levels?

Sustainability

• Did the intervention design include an appropriate sustainability and exit strategy (including promoting national/local ownership, use of national capacity, etc.) to support positive changes in the existence of regular, comparable and reliable gender statistics in the country at the end of the intervention?
• What is the likelihood that the benefits from the MEWGC in Ethiopia Programme be maintained for a reasonably long period of time if the programme were to cease?
• To what extent have MEWGC in Ethiopia national partners undertaken the necessary decision and course of actions to ensure the sustainability of the effects of the Programme?
• How has the programme enhanced ownership and contributed to the development of national capacity in order to ensure sustainability of efforts and benefits?
• How will the benefits of the intervention be secured for rights holders (i.e. what accountability and oversights
  systems were established)?

Ownership, inclusiveness and participation

• To what extent did the targeted population, citizens, participants, local and national authorities make the programme their own, taking an active role in it? What modes of participation (leadership) have driven the process?
• Have the stakeholders taken ownership of the programme? If so, how?
• To what extent and in what ways has ownership or the lack of it, impacted in the efficiency and effectiveness of the MEWGC in Ethiopia Programme?
• Was the designing, implementation and monitoring process of the programme participatory?
• Have stakeholders been given the chance to give their inputs? How useful was the process?

Gender Equality and Human Rights

• To what extent have gender and human rights considerations been integrated into the programme design and implementation?
• Have the programme interventions been following equality and non-discrimination principles at all time?

V. Scope of the evaluation

The midterm programme evaluation will cover the implementation of the programme from January 2019 – June 2020 under its three outcomes. This evaluation is a midterm assessment of the programme providing an evaluation on achievements as well as actionable recommendations for sustainability and improvement of future programme implementation. The evaluation will focus on all activities undertaken and geographical locations covered by the programme during the above-mentioned period.

The evaluation team is expected to establish the scope and limitations of the midterm evaluation, especially in terms of time frame, geographic coverage, thematic and programmatic coverage in addition to identifying which stakeholders will be included or excluded from the evaluation process. These will need to be discussed in the inception workshop. The evaluation team (one national consultant and one international consultant selected by UN Women) is expected to undertake a rapid evaluability assessment in the inception period and an initial assessment of the availability of secondary data in the country office necessary for the evaluation. Additionally, if the evaluation team will encounter any constraint – such as limited travel or accessibility to project sites - these limitations should be understood, and generalizing findings should be avoided where a strong sample has not been used. In addition, cultural aspects that could impact the collection of data should be analysed and integrated into data collection methods and tools. Evaluators are expected to include adequate time for testing data collection tools.

VI. Evaluation design (process and methods)

The evaluation should follow a formative approach and promote inclusion and participation by employing gender equality and human rights responsive approaches with a focus on utilization, empowerment or feminist approaches. The design should take into consideration that no comprehensive baseline document exists and that this information will first have to be gathered from various documents and resources (MEWGC in Ethiopia programme document, Annual Workplans, Implementing Partner Agreements, etc.).

The evaluation will be carried out following the UN Evaluation Group (UNEG) Norms and Standards (see http://www.unevaluation.org/document/detail/1914), UN Women Evaluation Policy as well as the Ethical Guidelines for evaluations in the UN system. The evaluation will also apply the UN Women GERAAS evaluation report quality checklist (https://www.unwomen.org/en/about-us/accountability/evaluation/decentralized-evaluations) , the UNEG Technical Note on the SWAP Evaluation Performance Indicator (https://www.unwomen.org/en/about-us/accountability/evaluation/un-coherence)  and the UNW Evaluation Handbook on Gender Responsive Evaluation (include link to UNW Evaluation Handbook:

https://www.unwomen.org/en/digital-library/publications/2015/4/un-women-evaluation-handbook-how-to-manage-gender-responsive-evaluation)  In line with Norms and Standards a management response will be prepared for this evaluation as practical means to enhance the use of evaluation findings and follow-up to the evaluation recommendations. The management response will identify who is responsible, what are the action points and deadlines.

Methods

The midterm evaluation methodology will be developed by the Evaluation Team and presented for approval to the Evaluation Reference Group. It should enable achievement of the evaluation purpose, be aligned with the evaluation design, address the evaluation criteria and answer the key questions through credible and gender-responsive techniques for data collection and analysis. Gender-responsive evaluation applies mixed-methods (quantitative and qualitative data collection methods and analytical approaches) that are appropriate to address the main evaluation questions, to account for complexity of gender relations and to ensure participatory and inclusive processes that are culturally sensitive and appropriate.

The evaluation is expected to apply a gender-responsive approach to assessing the contribution of the programme to development effectiveness. It should identify expected and unexpected changes in target and affected groups. It is anticipated that the evaluation will apply process tracing to identify the mechanisms of change and the probable contributions of the programme. The evaluation is expected to assess the strategic position of UN Women in MEWGC interventions in Ethiopia. It is anticipated that mixed qualitative and quantitative data will be collected, and case studies of different target groups will be developed, compared. The evaluation team will identify which factors, and which combinations of factors, are most frequently associated with a higher contribution of the programme to expected and unexpected outcomes.

The methods should include a wide range of data sources (e.g. documents, field information (collected by IPs during implementation and UN Women ECO during field visits), institutional information systems, financial records, beneficiaries, staff (from both IPs and UN Women), funders, experts, government officials and community groups). The evaluation team is particularly encouraged to use participatory methods to ensure that all stakeholders are consulted as part of the evaluation process. The evaluators are also expected to develop a sampling frame and specify how it would address the diversity of stakeholders.

Furthermore, due to possible travel restrictions, it is expected that the methods include creative options for virtual/online participation and data collection.

The evaluation team is encouraged to use a wide range of relevant participatory data collection tools, including:

• Interviews (including via skype, zoom)
• Focus group discussions (including through the use of skype, zoom)
• Secondary document analysis
• Observations (where travel is allowed)
• Multimedia (photography, drawing)
• Online Survey

The evaluators should take measures to ensure data quality, reliability and validity of data collection tools and methods and their responsiveness to gender equality and human rights. For instance, the limitations of the sample (representativeness) should be stated clearly and the data should be triangulated (cross-checked against other sources) to help ensure robust results.

In terms of data source, the following documents will be shared with the evaluation team by UN Women ECO:

• Programme Document of the three-year MEWGC in Ethiopia programme
• Ethiopia's second Growth and Transformation Plan (GTP II)
• United Nations Development Assistance Framework (UNDAF) 2016-2020
• Programme work plans and LoAs with IPs
• Progress reports (and presentations on progress and achievements)
• Donor reports
• IPs reports
• Publications and promotional materials
• Reports on specific activities
• Documents related to programme achievements
• Assessment reports on SDGs roll out and implementation
• UNW ECO Strategic Notes

VII. Stakeholder participation

The evaluators are expected to discuss during the Inception Workshops how the process will ensure participation of stakeholders at all stages:

• Preparation (inception workshop)
• Conduct (consultation of stakeholders; stakeholders as data collectors; interpretation)
• Reporting and use

The evaluators are encouraged to further analyse stakeholders' role according to the following characteristics:

• System roles (target groups, programme controllers, sources of expertise)
• Gender roles (intersections of sex, age, household and community roles)
• Human rights roles (rights holders, principal duty bearers, primary, secondary and tertiary duty bearers)
• Intended users and uses of the respective evaluation

The evaluators are encouraged to extend this analysis through mapping relationships and power dynamics. It is instrumental to a successful evaluation to focus in particular on participation of rights holders - especially women to ensure the application of a gender-responsive approach. It is also important to specify ethical safeguards that will be employed.

The evaluators are expected to validate findings engaging with stakeholders through workshops, debriefings or other forms of engagement.

[1] The first National Policy on Women was adopted in 1993 followed by consecutive legal and policy reforms including the adoption of the Revised Family Code and the Criminal Code in 2005.

[2] Federal Democratic Republic of Ethiopia Growth and Transformation Plan II (GTP II) (2015/16-2019/20) Target 8.

[3] Under the MDGs, Ethiopia achieved the goals on reducing the proportion of people living below the poverty line, significantly reducing the prevalence of hunger and undernourishment, expanding access to education, and narrowing the gap in school enrolment between boys and girls. In health, under-five mortality has been reduced by two thirds and substantial progress has been made in reducing HIV/AIDS, malaria, tuberculosis and other diseases. See: MDG 2014 Report ‘Assessing progress in Africa toward the Millennium Development Goals’ United Nations Economic Commission for Africa, African Union, African Development Bank and United Nations Development Programme, 2014 http://et.one.un.org/content/dam/unct/ethiopia/docs/MDG%20Report%202014%20(7).pdf.

Devoirs et responsabilités

Time frame

The evaluation will be done within 40 working days starting from the date of signing the consultancy agreement. A detailed work plan will be developed by the evaluation team during the inception phase based on inputs received from the Evaluation Reference Group.

IX. Expected deliverables

The evaluation manager (UN Women M&E focal point) and UNW Regional Evaluation Specialist will assure the quality of the evaluation report. The draft and final evaluation report will be shared with the Evaluation Reference Group and the Evaluation Management Group for quality review. The final report will be approved by the evaluation management group. The final evaluation report will be independently assessed using quality standards outlined in the UNW Global Evaluation Reports Assessment and Analysis System (GERAAS). The final evaluation report and evaluation management responses will be publicly disclosed in the UNW GATE system.

The final report should follow the standard table of contents for an evaluation report:

I) Title and opening pages

II) Executive summary

III) Background and purpose of the evaluation

IV) Programme description and context

V) Evaluation objectives and scope

VI) Evaluation methodology and limitations

VII) Findings

VIII) Conclusions

IX) Recommendations

X) Lessons Learned

ANNEXES: ToR; Documents consulted; lists of institutions/stakeholders interviewed or consulted and sites visited; analytical results and methodology related documentation, such as evaluation matrix; list of findings and recommendations.

X. Management of evaluation

At UN Women the evaluation phases are:

Stage 1: Planning

Stage 2: Preparation: This includes the stakeholder analysis and establishment of the Reference Group, Evaluation Management Group, development of the ToR, and recruitment of the evaluation team

Stage 3: Conduct: Inception workshop, data collection and analysis

Stage 4: Reporting: Presentation of preliminary findings, draft and final reports

Stage 5: Use and follow up: Management response, dissemination of the report, and follow up to the implementation of the management response

The evaluations will have the following management structures:

Country Office M&E Analyst (Evaluation Manager): for coordination and day-to-day management with support from Coordination team;

Evaluation Management Group for administrative support and accountability: Country Representative or Deputy Country Representative, M&E Officer, Regional Evaluation Specialist (in advisory capacity), National Programme Coordinator.

Evaluation Reference Group for substantive technical support: UN Women programme staff, national government partners, development partners/donors, UNCT representatives, former and current UN Women Civil Society Advisory Group Members and CSO partners. These should be selected based on the stakeholder analysis.

The main roles and responsibilities during the process are:

Compétences

Required competencies for both International / National consultant

• Strong knowledge of issues concerning development frameworks, statistics, gender statistics, gender equality and women's empowerment.
• Experience in working with UN agencies and UN programmes and evaluations.
• Excellent facilitation and communication skills.
• Experience with focus group discussions and key informant interviews.
• Ability to deal with multi-stakeholder groups; as well as displaying cultural sensitivity.
• Ability to write focused evaluation reports; as well as excellent writing skills (in English).
• Broad experience in quantitative and qualitative data collection methods.
• Experience and skill in administering online data collection and facilitating virtual meetings.
• Experience in undertaking home-based consultancies.
• Willingness and ability to travel to project sites, when deemed necessary.
• Ability to work in a team.

Core values / guiding principles

The evaluators will adhere to the following core values and guiding principles:

• Integrity: Demonstrating consistency in upholding and promoting the values of UN Women in actions and decisions, in line with the UN Code of Conduct.
• Cultural Sensitivity/Valuing diversity: Demonstrating an appreciation of the multicultural nature of the organization and the diversity of its staff. Demonstrating an international outlook, appreciating differences in values and learning from cultural diversity.

Qualifications et expériences requises

XI. Evaluation team composition, skills and experiences

An international evaluation consultant supported by a national evaluation expert will undertake the evaluation. The evaluation team will be assembled to ensure the right mix of evaluation expertise, knowledge of the national context and expert knowledge of statistics in general and gender statistics in particular. UN Women will maintain Gender balance of the consultants.

Required Background and Experience