Background

Justification for the consultancy

In order to further develop the Evidence Hub there is a need for short-term consultancy support from an evaluation expert to conduct a meta-evaluation and discrete meta-analysis of the evidence generated from a selection of final external evaluations completed between 2017 and 2019. It is necessary to seek external support to complete this review as a stand-alone piece of work for the following reasons: 

1. the potential evidence covers nearly 100 evaluation reports received between 2017 and 2019 (beyond the capability of the current team);
2. the UN Trust Fund is committed to content generation informed by quality assured material and credible data;
3. this exercise will enable the UN Trust Fund to determine whether actions taken to advance its evaluation practice (since the previous meta-evaluation in 2016) have been enough to improve the quality of evaluations commissioned by grantee organizations overall;   
4. it will also enable the team to access and build a database of coded evaluations that can be analyzed by key criteria in the future;
5. finally the analysis will allow for independent verification of projects that have been assessed as effective and sustainable to extract evidence of what works in the field of EVAW/G for knowledge generation and dissemination.

Methodology

To align with UN Women practice, this assessment will adapt the Global Evaluation Report Assessment and Analysis System (GERAAS) Methodology. It utilizes the UNEG evaluation reports standards as a basis for review and assessment of final evaluation reports, while ensuring specific standards are relevant to UN Women. The four main objectives are to:

• improve the quality and utility of evaluation reports;
• strengthen internal capacity on gender responsive evaluation;
• improve [UN Women’s][1] performance and organizational effectiveness; and
• promote learning and knowledge management.

For UNTF purposes, this adapted process will consist of four main parts for the meta-evaluation: (1) basic information and classification of the evaluations; (2) assessment of the quality against each of the eight parameters; (3) sub-coding within each of the eight parameters[2]; and (4) assessment and rating of the overall quality of the entire individual evaluation report. For the meta-analysis step three (3) of the process above should include further sub-coding – to be devised with the UN Trust Fund team - to inform further meta-analysis projects (this consultancy will be expected to report on % of projects assessed as effective and sustainable, at minimum).

All these steps should be completed with active participation of the UN Trust Fund’s M&E team to enable the Secretariat to apply the methodology again in the future. Any tools used, such as qualitative assessment software packages, must be transferable to the UN Trust Fund including all the tagging/coding conducted to ensure the data, analysis and approach can be further interrogated and applied again

[1] In the context of the UN Trust Fund, this relates to grantee organizations.

[2] The selected candidate must have access to software for qualitative analysis, as well as the capabilities to hand over coding and scoring upon completion of the consultancy.

Duties and Responsibilities

The overall objective of the consultancy is to produce a meta-evaluation and (discrete) meta-analysis of evaluations produced by UN Trust Fund grantees from 2017 through 2019 and prepare a database of coded material to enable future analysis. The specific outputs will be:

1. Before 15 May 2019: An inception report describing the approach to the project, including the software/tools to be utilized, and establishing the code dictionary for qualitative data analysis.
2. By 30 June 2019 a draft (partial) report on initial findings (based on a qualitative analysis tool with coded evaluations) and by 31 July 2019 a final report to include:
3. An assessment of the quality of evaluations produced by UN Trust Fund grantees against UN Women / GERAAS and UNEG quality standards including a summary of trends in quality and credibility of evaluations (e.g. by variations in quality by theme, region or evaluation design). The report should also include recommendations for the UN Trust Fund to improve evaluation practice and identify a selection of satisfactory quality evaluations which can be used for UNTF’s next objective;
4. An analysis of a sub-set of evaluations (those of satisfactory or above quality) to identify those assessed as effective and sustainable projects to inform further synthesize of evaluative knowledge.
5. Handover of database of coded material to the UN Trust Fund. This would be aligned with the code dictionary generated during the inception phase. Any changes or additions to the dictionary should be explained in narrative form.

Activities (exact order to be agreed in the inception phase)

1. Record and map the evaluations produced by UN Trust Fund grantees by theme, country, region and evaluation design – in a selected qualitative software tool that can be handed over to the UNTF.
2. Identify what exclusion criteria to apply to the evaluation universe.
3. Develop a code dictionary for the UNTF to utilize for future analysis of evaluation reports.
4. Assessment of the quality of individual reports and scoring:
   1. This comprises an assessment of the quality of individual evaluation reports against (at minimum) eight UN Women adapted UNEG Parameters (Object and Context of Evaluation; Evaluation Purpose, Objectives and Scope; Evaluation Methodology; Findings; Conclusions and Lessons Learned; Recommendations; Gender and Human Rights Considerations; and the report structure).
   2. Sub-code within each parameter – and further parameters as agreed with the UN Trust Fund team - utilizing the code dictionary developed during the inception stage.
5. Draft a report on the findings from the meta-evaluation and recommendations for the UN Trust Fund to: (1) improve evaluation practice; and (2) provide targeted support to grantees on evaluation management.
6. Identify a sub-set of the satisfactory (or above) quality evaluations to conduct a discrete meta-analysis of the evidence produced (for example 50 evaluations overall), on which projects were assessed as effective and sustainable.

Inputs and Timing

Contribution from the beneficiary (UNTF Secretariat): the UNTF will provide the consultant with access to all the documentation required, guidance, direction and the agreed daily rate upon delivery of the work delivered as above.

Consultants input: over a period of three months (15 May to end July 2019) to deliver the following products:

• Inception report - 15 May 2019
• First draft report (and database/tool) - 30 June 2019
• Final report (and database/tool to be handed over to the UN Trust Fund) - 31 July 2019

The consultant is expected to cover the costs of the home-based activities, the initial database/tool cost and any assistance needed within the fee rate.

Reporting

The consultant will report to the UNTF’s Monitoring, Evaluation and Knowledge Manager, Gemma Wood. The draft report should be shared with the UNTF Secretariat by 30 June 2019 for consultation and the final report to be completed by 31 July 2019.

Competencies

Required skills and experience

Core Values:

• Respect for Diversity;
• Integrity;
• Professionalism.

Core Competencies:

• Awareness and Sensitivity Regarding Gender Issues;
• Accountability;
• Creative Problem Solving;
• Effective Communication;
• Inclusive Collaboration;
• Stakeholder Engagement;
• Leading by Example.

Please visit this link for more information on UN Women’s Core Values and Competencies: http://www.unwomen.org/-/media/headquarters/attachments/sections/about%20us/employment/un-women-employment-values-and-competencies-definitions-en.pdf

Required Skills and Experience

Education:

• Master’s degree or equivalent in social sciences, human rights, gender/women's studies, international development, or a related field is required.

Experience:

• A minimum of 5 years relevant experience undertaking evaluations is required including proven practical professional experience in designing and conducting major evaluations;
• Substantive experience in evaluating similar development projects related to local development and political and economic empowerment of women;
• Substantive experience in evaluating projects and programmes with a strong focus on ending violence against women and girls and/or women’s empowerment and gender equality is preferred;
• Excellent and proven knowledge of evaluation methodologies and approaches;
• Experience with qualitative software;
• Experience with meta-evaluation and meta-analysis of evaluation reports, preferably with UN agencies, is an asset;
• Familiarity with UNEG evaluation standards, is an asset;
• Proven experience in producing coherent, clear analytic reports and knowledge pieces is a requirement.

Language:

• Full proficiency in English (written and spoken) required;
• Working language of at least one of the following languages is also required: Spanish a/o French.

How to apply

Applicants should provide a CV and a statement of interest included in a 2-page proposal setting out an approach to the work and proposed budget. The selected consultant may suggest that additional support is required in the form of an assistant or data specialist, however this additional support must be organized and managed by the selected consultant within the daily rate agreed. Note that all applications must include (as an attachment) the completed UN Women Personal History form (P-11) which can be downloaded from http://www.unwomen.org/about-us/employment. Kindly note that the system will only allow one attachment, hence the need to add your statement of interest into the same document as the signed P-11. Applications without the completed UN Women P-11 form will be treated as incomplete and will not be considered for further assessment. Deadline for submission is 5 April 2019.

Due to the large number of applications we receive, we are only able to inform the successful candidates about the outcome or status of the selection process.

Ethical code of conduct

It is expected that the consultant will ensure that the confidentiality and independence of judgment are maintained, and that findings and recommendations are independently presented. The consultant will operate in an impartial and unbiased manner and give a balanced presentation of strengths and weaknesses of the issues being assessed. The consultant must disclose in writing any experience, which may give rise to a potential conflict of interest, and to deal honestly in resolving any conflict of interest which may arise.