Background

With the emphasis on promoting organizational learning and accountability through evaluative evidence, IEAS has performed a series of synthesis of results of evaluations. The annual meta-synthesis systematically analyzed insights and findings from a series of evaluations and helped to inform corporate processes, polices and strategies as well as external assessments by donors. A common analytical framework has been used over the years to ensure consistency in the assessment process and establish trends against a set of predefined rubrics aligned with the operational and development priorities of the previous Strategic Plan (2013-2017).  

This is the first synthesis covering the period of the UN Women Strategic Plan (2018-2021). While it is complementary to previous analyses, the analytical framework has been revised cognizant of the institutional maturity and growth of UN Women and in sync with the priorities of the current Strategic Plan. The synthesis will in particular draw insights from ten CPEs, five regional and thematic evaluations, two global evaluations, one corporate evaluation, four joint evaluations as well as 17 programme and project evaluations. In total 37 evaluations. All regions will be represented availing insights on a range of contexts and scales of UN Women’s normative, coordination and operational responses.

Duties and Responsibilities

Purpose

The meta-synthesis will contribute to learning through the identification of recurring findings, lessons learned and evidence on the key areas of UN Women’s work and provide analysis that can contribute to the forthcoming Strategic Plan mid-term review and other corporate and country level processes. The synthesis will also seek to capture internal and external enablers that drive or impede progress towards the achievement of gender equality and women’s empowerment.

Methods

This meta-synthesis will solely be based on a desk review of corporate and decentralized evaluation reports. The revised analytical framework takes at its heart the complementarity of the UN Women’s integrated mandate spanning normative support, UN coordination and operational activities; commonly found features related to organizational effectiveness and efficiency results; and internal and external factors that enable or inhibit performance. Nonetheless, the depth of the analysis on certain themes is contingent upon the coverage, clarity and strength of evidence available in the body of the evaluation reports.

Scope of the consultancy

The meta-synthesis will be conducted by an external consultant over a 2-month period. The primary method for this synthesis is a desk review of 37[1] evaluation reports completed in 2017/2018.

Key deliverables

The main output of this consultancy is a synthesis of the evaluation reports (maximum 30 pages report). This requires analysis of the findings, conclusions and recommendations including lessons learned and good practices presented in all evaluation reports.

Proposed approach

The approach will look across the body of evaluation reports rated ‘fair and above’; identifying lessons emerging most strongly; and highlight examples of good practices on what has worked, what has not, and why. As most evidence in the body of evaluation reports is structured against the standard evaluation criteria, the analysis will be performed against relevance, effectiveness, efficiency, sustainability and impact.

Specifically, the synthesis will involve quantitative and qualitative analysis.

Quantitative analysis:

Based on the level of evidence available, the qualitative analysis would be undertaken in accordance with pre-established criteria of the OECD DAC approach for conducting development effectiveness reviews to be provided.

• The quantitative analysis will consider evaluation insights according to: i) standard evaluation criteria - relevance, effectiveness, efficiency, sustainability and impact (whenever this criterion was covered) and (ii) UN Women integrated mandate and Strategic Plan 2018-2021) operational effectiveness and efficiency results.
• Assessment of coverage of evidence: assessment of whether the coverage of evidence against each sub-criteria in the evaluation reports is strong, moderate or weak (coverage).
• The results for each sub-criterion – including the quantitative proportion of evaluations will be classified as highly unsatisfactory, unsatisfactory, satisfactory or highly satisfactory for each of the five major criteria and their associated sub-criteria. Percentages will only be provided for those evaluations which addressed a given sub-criterion.
• The analysis will examine characteristics of high performing and low performing areas to try to examine what element and a combination of elements contribute to greater organizational effectiveness.
•  To be consistent, the synthesis will use terms “most”, “many”, “some” and “few” to describe the  frequency with which an observation was noted, as a percentage of the number of evaluations addressing the sub-criterion.
• The overall framework and measurement for main development effectiveness criteria and associated sub-rubrics to be provided.

Qualitative analysis

The analysis will apply a structured qualitative analysis against the UN Women Strategic Plan (2018-2021) Operational Effectiveness and Efficiency Priorities and Principles. 

The identified lessons will be summarized and grouped under broad themes as outlined in the table below. Specific examples of good practices will be provided throughout the report. In particular, the synthesis will capture:

• High level drivers, both internal and external, that enable or hinder the performance of UN Women;
• High level emerging findings on leveraging the integrated mandate and findings on innovative approaches;
• Identification of compelling stories or illustrative examples to showcase the level of change registered at various levels including SP thematic areas.

Management Of the Consultancy

The UN-Women Independent Evaluation Service is responsible for the management of the meta-synthesis. The IES will provide support to assure the quality of the draft and final products as well as facilitating administrative and other backstopping support.

The selected individual will assume sole responsibility in ensuring the consistency, quality and timely delivery of expected products, and overall coordination with UN-Women IES. An audit trail of comments for the draft and final report will be maintained by the consultant.

[1] The total number of evaluation reports to be considered for the synthesis is 32. Quality of all reports were externally assessed and judged to have met the minimum quality standards established by UN Women IES. A large majority of the reports were written in English (21) while the remaining 7 are Spanish and 6 French.   

[2] Upon finalization the draft report, the database will be provided to UN Women IES.

Competencies

Functional Competencies:

• Strong analytical skills, including ability to rapidly analyze and integrate diverse information with a discerning sense for quality of data;
• Good mastery of information technology required for organized presentation of information;
• Proven experience with meta-evaluation and meta-analysis of evaluation reports, preferably with UN agencies and development partners.
• Consistently approaches work with energy and a positive, constructive attitude;
• Proven organizational skills and ability to handle a large volume of work; and
• Strong oral and written communication skills.

Corporate Competencies:

• Demonstrates integrity by modelling UN values and ethical standards, including UNEG norms;
• Acts as a team player and exercises diplomacy, tact and politeness; and
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Required Skills and Experience

Education

• A post-graduate degree or higher in a relevant field

Experience

• Proven experience and background in gender equality/gender analysis
• Experience with meta-evaluation and meta-analysis of evaluation reports, preferably with UN agencies, preferred
• Excellent and proven knowledge of evaluation methodologies and approaches, including specifically gender-responsive evaluations
• Proven practical professional experience in designing and conducting complex multi-country evaluations
• Familiarity with UNEG evaluation standards, including the UNEG Guidance on Integrating Human Rights and Gender Equality in Evaluations, is an asset

Languages

• Excellent analytical and writing skills in English, Spanish and French are required.

Application Instructions

• Personal CV or P11, indicating all experience from similar projects, as well as the contact details (email and telephone number) of at least three (3) professional references;
• A brief cover letter (250-400 words) explaining your interest in this assignment and a lumpsum amount to carry out this consultancy (inclusive of all expenses). Please note this is a homebased assignment, thus no travel is required.