Background

Purpose and Use of the Joint Evaluation

As the EmPower Project is approaching the end of the project timeframe, an independent final evaluation will be undertaken during the last year of the EmPower project, as per EmPower’s Project Document and Programme Cooperation Agreement. It will be a joint evaluation between UN Women and UNEP, in accordance with the guidance from UN Evaluation Group, and involve the evaluation units of both organizations following their evaluation policies. The purpose of the evaluation is to feed into learning about what worked well with respect to the joint approach to gender-responsive rights-based climate change resilience and DRR and what can be improved, and will serve accountability purposes, it will also feed into decision-making regarding a potential phase II of the project. The primary evaluation users (UN Women and UNEP Regional office for Asia and the Pacific) will use the evaluation to further strategize for gender-responsive climate actions. It will be also used by the EmPower team and stakeholders to design the possible Phase II of the project. Secondary users within the respective organizations and partners will use the information to learn about what works to integrate gender in climate change and disaster risk reduction approaches. The donor SiDA may use the evaluation as input for decision-making purposes.

Objectives of the Joint Evaluation

The overall objective of the joint evaluation is to  to assess progress made over the project period towards the attainment of the intended outcome “countries in Asia and the Pacific are implementing gender-responsive climate change and DRR actions to address key drivers of gender-based vulnerabilities,” and the relevance, effectiveness, efficiency, sustainability, human rights, and gender equality, including a look into how the most vulnerable groups (focusing on women, including those living in remote disaster prone rural areas in the EmPower project’s context)were engaged in the project. It will also assess environmental safeguards taken by the project and contributions towards impact. It should also provide an assessment of how EmPower has integrated the recommendations and lessons learned from the Mid-term Review. It will also provide inputs and give guidance for the potential Phase II of the EmPower Project. The performance of the project will be assessed against the indicators presented in the results and reporting framework.  The joint evaluation will:

1. Assess the relevance and UN system coherence with respect to programme design and implementation;
2. Assess the effectiveness and organizational efficiency of the approaches implemented in attaining the intended results;
3. Assess the contribution towards impact, unintended consequences, potential for sustainability, and integration of human rights and gender equality in design and implementation; and
4. Produce lessons learned and issue actionable recommendations for the potential EmPower phase II or similar programming.

Key Evaluation Questions

The evaluation team will revise the evaluation questions based on consultations during the inception phase and considering the feasibility of objectives and scope outlined in this Terms of Reference. The evaluation team should raise and address any other relevant issues that may emerge during the evaluation. They should be guided but not limited by the evaluation questions listed below. The evaluation team will develop an evaluation matrix during the inception phase in consultation with the Evaluation Management Group (EMG) and the Evaluation Reference Group (ERG) which will outline the questions and means of answering them. Please see the UNEP Evaluation Criteria (annexed) with suggestions on how to further articulate the evaluation questions.

[1] Please refer to the Integrating Human Rights and Gender Equality in Evaluation for examples of questions (see Table 11 on pp.81-85): link

Scope of the Joint Evaluation

The evaluation will cover all components of the programme, including those implemented by each participating UN Agency.

Time Frame: the evaluation will cover the entire project life (from April 2018 up to quarter 2 of 2022 as the last year of the project).

Geographical coverage: the evaluation will focus on activities implemented in Bangladesh, Cambodia, and Viet Nam, as well as at the regional level.

Stakeholder coverage: the evaluation will reach out to stakeholders, i.e. beneficiaries, participating governments, civil society partners, implementing partners at the national and regional levels, and partner agencies, as well as the project steering committee members and project partners.

Limitations: there might be possible limitations on the travel to EmPower’s three countries, Bangladesh, Cambodia, and Viet Nam, considering the dynamic situation due to the COVID-19 pandemic. Close collaboration with national evaluators and the conduct of virtual interviews/meetings would be required. Triangulation of information received from different sources and synthesis of key findings across the different countries and components will feed into the overall findings, but generalizations will not be made.

Design of the Joint Evaluation

The evaluation will be, in its nature, summative of the entire project[1] period (up to Q2 2022) and include recommendations for the potential Phase II. The approach should also promote inclusion and participation by employing gender equality and human rights responsive approaches with a focus on utilisation[2], empowerment[3] or feminist approaches[4]. The evaluation will be gender-responsive which applies mixed-methods (quantitative and qualitative data collection methods and analytical approaches) to account for complexity of gender relations and to ensure participatory and inclusive processes that are culturally appropriate.  The design of the joint evaluation should be theory-based and the Theory of Change of the EmPower project should be used as the basis for the evaluation[5].

[1] The evaluation can refer to UNEP Glossary of project terms.

[2] Promotes intended use by intended users. Strong focus on participation of users throughout the evaluation process.

[3] Project participants are involved in the conduct of the evaluation. An outside evaluator serves as a coach or facilitator in the evaluation process.

[4] Addresses and examines opportunities to reverse gender inequities that lead to social injustice. Prioritizes women’s experience and voices, including women from discriminated and marginalized groups.

[5] Please refer to the UNEP guidance on Theory of change in evaluation to be provided to the consultant team.

Methodological approach

The evaluation will be, in its nature, summative of the entire project period (up to Q1 2022) and include recommendations for the potential Phase II. The approach should also promote inclusion and participation by employing gender equality and human rights responsive approaches with a focus on utilisation[1], empowerment[2] or feminist approaches[3]. The evaluation will be gender-responsive, which applies mixed-methods (quantitative and qualitative data collection methods and analytical approaches) to account for the complexity of gender relations and to ensure participatory and inclusive processes that are culturally appropriate. The design of the joint evaluation should be theory-based, and the Theory of Change of the EmPower Project should be used as the basis for the evaluation, which will be reconstructed through a theory of change workshop with the programme team. The evaluation methodology should enable achievement of the evaluation purpose, be aligned with the evaluation approach, and be designed to address the evaluation criteria and answer the key questions through credible techniques for data collection and analysis.

The suggested methods of data collection include desk review, key informant interviews, focus group discussions, and survey. A case study approach will be taken to allow for in-depth look at key issues or implementation modalities at country level, which will include stakeholder consultation, observation, and review and documentation analysis (e. g. progress and completion reports, workshop and mission reports, knowledge and advocacy products, and other appropriate documentation produced and related by UN Women and UNEP). The criteria for case study selection will be identified during the inception phase – it is likely that the case studies will be limited to two countries and national consultants in the selected countries will be engaged to lead the data collation. The evaluation must integrate gender and human rights approaches and perspectives throughout data collection and analysis. It is particularly important to understand and assess how the project addresses complex, intersectional discrimination and how this affects women’s rights.

Evaluators will conduct consultation with stakeholder groups, to the extent possible, using participatory tools and suggest a plan for inclusion of women and individuals and groups who are vulnerable and/or discriminated against in the consultation process and a plan for translation, as necessary. It would include women-led CSOs, indigenous community groups, LGBTIQ+ communities, persons with disabilities, and women entrepreneurs in the context of the EmPower Project. Based on consultations, the national consultants will visit selected project sites to validate the findings of the desk review and documentation analysis, and identify good practices and lessons learned. The evaluation may employ a participatory storytelling or most significant change approach through the country case study visits.

The entire evaluation will be undertaken as per UNEG guidelines and consider a human-rights-based and gender empowerment approach[4]. The evaluation experts and all their direct collaborators will follow UN Women’s Evaluation Handbook[5] UNEP guidance and UNEG Ethical guidelines.

[1] Promotes intended use by intended users. Strong focus on participation of users throughout the evaluation process.

[2] Project participants are involved in the conduct of the evaluation. An outside evaluator serves as a coach or facilitator in the evaluation process.

[3] Addresses and examines opportunities to reverse gender inequities that lead to social injustice. Prioritizes women’s experience and voices, including women from discriminated and marginalized groups.

[4] UNEG Norms and Standards for Evaluation (2016): https://www.betterevaluation.org/en/resources/example/UNEG-evaluation-standards-2016; UNEG Ethical Guidelines for Evaluations (2020): http://www.unevaluation.org/document/detail/2866; Integrating Human Rights and Gender Equality in Evaluation (2014): http://www.uneval.org/document/detail/1616

[5] UN Women’s Evaluation Handbook https://genderevaluation.unwomen.org/en/evaluation-handbook

Limitations

Given the ongoing COVID-19 pandemic, the use of participatory methods may be limited, and travel restrictions may limit the possibility of in-person data collection by the team leader. UN Women and UNEP will monitor the situation and during the inception phase will determine the way forward. The evaluation team will rely on the accuracy and completeness of the provided documents by the Offices, with independent verification of the information provided, where possible. To avoid biases raised, the information will be triangulated and validated with the Offices and Evaluation Management and Reference Groups.

Stakeholder participation

There will be several stakeholder meetings, including debrief meetings with key in-country stakeholders (at minimum, UN Women, and UNEP) at the end of each field visit to validate emerging findings from the mission and identify/fill data gaps. Then, once the evaluators have analysed the data, they will present preliminary findings to the Evaluation Reference Group (ERG) to validate these and consider preliminary feedback in the development of the draft report.

Key stakeholders (e.g., internal stakeholders, programme/project partners, donors, the Project Steering Committee, etc.) will be consulted through this the joint evaluation. It is important to pay particular attention to the participation of rights holders—women and vulnerable and marginalized groups—to ensure the application of a gender-responsive approach – this will be done through a civil society representative on the Evaluation Reference Group (ERG) and through the case studies.

• Stakeholder participation in data collection: there will be the inclusion of a diversity of stakeholders – including vulnerable groups – in data collection. If possible, persons with disabilities should also be consulted. Ethical safeguards will be taken and safety protocols in line with WHO Guidelines on research on violence against women and in line with safety recommendations with respect to COVID-19.

Evaluation Phases

The joint evaluation will be conducted according to the following tentative timeline and with the main deliverables outlined below. The SSA will be engaged in the stage 1-3 (February to June 2022) for approximately 45 days of work (this is an indicative number that should be used as a guideline).

STAGE 0: PREPARATION (-mid February 2022)

• Joint review (UN Women and UNEP) of the Terms of the Reference
• Formation of the Evaluation Management Group (EMG) and the Evaluation Reference Group (ERG)
• Recruiting an evaluation team including one international and 2 national consultants

STAGE 1: INCEPTION (-early March 2022)

• Briefing and consultation with EMG
• Desk review of key documents
• Theory of change workshop
• Inception report including the evaluability assessment, stakeholder mapping, theory of change (reconstructed as necessary), methodology, workplan, evaluation matrix, and data collection tools
• Presentation of the inception report to ERG

STAGE 2: CONDUCT (-early Apr 2022)

• Data collection, including virtual and on-site interviews and meetings and debriefing of UN Women upon finalization
• Data systematization, analysis and interpretation of findings

* Due to the COVID pandemic, the option of travelling or doing virtual consultations will be discussed during the inception phase and consider the latest development on the pandemic.

STAGE 3: REPORTING (-early Jun 2022)

• Present the preliminary findings to EMG and ERG to validate findings and allow the evaluators to incorporate preliminary feedback in the draft report
• Draft report
• Comments and feedback from Evaluation Management and Reference Groups tracked for transparency
• Final evaluation report
• Presentation of the final evaluation report in the Annual Review Meeting of EmPower (June 2022).

STAGE 4: DISSEMINATION (-early Nov 2022)

• Communications based on the final evaluation include the UNEP assessments to be disseminated widely to stakeholders and public
• Management response within 6 weeks of completion.

Duties and Responsibilities

Duties and Responsibilities

Under the direct supervision of the Regional Evaluation Specialist and the Regional Adviser on Gender and Climate Change of UN Women ROAP, and in consultation with the Evaluation Management Group an independent evaluation expert will be hired to undertake the Project’s Mid-Term Review. The expert will be responsible for the following:

1. Lead the evaluation team, which will include 2 national consultants who are based in the EmPower pilot countries to collectively conduct data collection/analysis and generate the evaluation products.
2. Perform desk review of the project’s key documents including Program Document, Inception Report, Annual Work Plans and Budget, all progress reports of UN Women, UN Environment and implementing partners, all knowledge products including studies, research and outcome documents from all conferences and workshops that have been conducted within the framework of the project.
3. Prepare an Inception Report detailing the evaluation team’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures.
4. Collect primary data, including interviews with the project team members and key stakeholders involved in the project at regional level and based in Bangladesh, Cambodia and Viet Nam. UN Women will provide a list of stakeholders and will assist with liaising and coordinating the meeting schedules as needed.
5. Analyse the primary and secondary data in line with the objectives and key questions of the evaluation to identify findings, conclusions and recommendations as detailed in the evaluation report.
6. Present the preliminary findings to EMG and ERG to validate and integrate feedback in the draft report.
7. Prepare a draft evaluation report and integrate feedback received from ERG and EMG into the final evaluation report not exceeding 40-50 pages (with annexes not to exceed 50 pages) in word format.
8. Prepare a presentation of the final evaluation conclusions and recommendations.

Expected deliverables

* Payment will be made upon submission of deliverables with an approval of the Regional Evaluation Specialist and Regional Adviser on Gender and Climate Change

* All deliverables should be in line with the UN Women Editorial Style Guide and the EmPower Visual Identity Guide for Publications.

* All deliverables should be written and generated in English. Data collected is property of UN Women if so requested.

Management of the joint evaluation

The joint evaluation will be jointly managed by both UN Women and UNEP. UN Women and UNEP would support the evaluators in the evaluation process, including preparation, conduct, reporting, however, should not interfere with the impartiality of the evaluation. UN Women and UNEP will support the logistical support needed, such as materials and office space (only if the situation of COVID-19 at the time of the evaluation allows). UN Women and UNEP would jointly engage in the planning and reporting stages. The joint evaluation will have an Evaluation Management Group (EMG) and an Evaluation Reference Group (ERG) to facilitate the management of the evaluation.

The joint evaluation will have the following groups:

• Co-managers: UN Women Regional Evaluation Specialist, Independent Evaluation Service, will take the lead in co-managing the evaluation in close coordination and consultation with UNEP Independent Evaluation Office. The evaluation co-managers will ensure the independence and impartiality of the evaluation process given that they are not engaged in the management or implementation of the programme. They will provide guidance on methodology and oversee the quality assurance; and serve as the main contact with the evaluation team, but will work closely with the Evaluation Management Group to manage the logistics.
• Evaluation Management Group (EMG): the EMG includes the evaluation managers and programme personnel from UN Women and UNEP. It oversees the day-to-day management of the evaluation, also provides logistics and other types of support. The EMG is responsible for providing overall quality assurance on the evaluation process and deliverables.
• Evaluation Reference Group (ERG): In addition, the joint evaluation requires an Evaluation Reference Group (ERG) to ensure that the voices of stakeholders are heard and responded to through the evaluation process. The ERG will review evaluation products for factual correctness, possible misinterpretations, or major gaps. It is important to ensure their ownership of the evaluation process, as they are also representing the organizations that will carry forward the recommendations. The ERG should be limited to 6-8 people max and represent the diverse range of stakeholders from CSOs, UN agencies, governments, and the donor. 
• Evaluation team: A 3-member evaluation team is proposed. One International team leader with expertise in evaluation and with experience evaluating climate change and disaster risk reduction initiatives from a gender perspective; the team leader is responsible for the overall evaluation and ensuring quality of the evaluation products; and two national consultants (one each in two of the programme countries) will be engaged to lead the case studies in country and support the team leader with data collection and analysis; they should have experience in evaluation; the co-managers along with the UN Women Project Manager will recruit and manage the consultants, yet the evaluation team leader will liaise directly with the national consultants and will be responsible for the final evaluation products.

Ethical code of conduct

UN Women has developed a UN Women Evaluation Consultants Agreement Form for evaluators that must be signed as part of the contracting process, which is based on the UNEG Ethical Guidelines and Code of Conduct. These documents will be annexed to the contracts. All data collected by the evaluation team members must be submitted to the evaluation manager in Word, PowerPoint or Excel formats and is the property of UN Women.  Proper storage of data is essential for ensuring confidentiality and a data protection plan will be developed during the inception phase. The evaluation’s value added is its impartial and systematic assessment of the programme. As with the other stages of the evaluation, involvement of stakeholders should not interfere with the impartiality of the evaluation. The evaluator(s) have the final judgment on the findings, conclusions and recommendations of the evaluation report, and the evaluator(s) must be protected from pressures to change information in the report. Proper procedures for data collection with rights holders who may have been affected by violence must be adhered to as outlined in the WHO Ethical and Safety Recommendations for research on violence against women. Additionally, if the evaluator(s) identify issues of wrongdoing, fraud or other unethical conduct, UN Women and UNEP procedures must be followed and confidentiality be maintained. The UN Women Legal Framework for Addressing Non-Compliance with UN Standards of Conduct, and accompanying policies protecting against retaliation and prohibiting harassment and abuse of authority, provide a cohesive framework aimed at creating and maintaining a harmonious working environment, ensuring that staff members do not engage in any wrongdoing and that all allegations of wrongdoing are reported promptly, investigated and appropriate action taken to achieve accountability.

Competencies

Core values:

• Integrity: Demonstrate consistency in upholding and promoting the values of UN Women in actions and decisions, in line with the UN Code of Conduct.
• Professionalism: Demonstrate professional competence and expert knowledge of the pertinent substantive areas of work.
• Cultural sensitivity and valuing diversity: Demonstrate an appreciation of the multicultural nature of the organization and the diversity of its staff. Demonstrate an international outlook, appreciating difference in values and learning from cultural diversity.

Corporate competencies:

• Awareness and sensitivity regarding gender issues;
• Creative problem solving;
• Effective communication;
• Inclusive collaboration;
• Stakeholder engagement;
• Demonstrates integrity by modelling the United Nations' values and ethical standards;
• Promotes the vision, mission and strategic goals of the UN and UN Women;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability;
• Ability and willingness to work as part of a team to meet tight deadlines and produce high quality work.

Required Skills and Experience

Academic qualifications:

• Master’s degree in relevant discipline or Bachelor’s degree + 2 additional years of relevant work experience (e.g., social sciences, international development, gender studies, sustainable development and social studies, environmental governance/management and science, etc.)

Essential knowledge and experience:

• A minimum of 5 years of experience leading evaluations, including evaluations of multi-stakeholder projects for multilateral organizations is required;
• Extensive knowledge of qualitative and quantitative evaluation methods is required;
• Extensive experience of Gender equality and Environmental or Climate Change or Disaster Risk Reduction or Renewable Energy work within development cooperation preferred.
• Knowledge in results-based programming in support of gender justice and human rights in the context of climate change and DRR is highly desirable;
• Proven experience in drafting and writing to produce and present concise and analytical reports;
• Working experience in the Asia and the Pacific is preferred;
• Previous working experience in Bangladesh or Cambodia or Viet Nam would be considered as an asset;
• Knowledge in climate change or DRR policy in Bangladesh or Cambodia or Viet Nam is an asset;
• Experience working with the UN system a strong asset;
• Excellent English writing and communication skills are required.

Submission of application:

Interested candidates are encouraged to submit an electronic application to hr.bangkok@unwomen.org with CC to vanissa.limkriangkrai@unwomen.org not later than 10 January 2022.

The submission package includes:

• Cover letter
• Sample of at least two evaluation reports, preferably in the areas of gender, environmental, climate change, disaster risk reduction or renewable energy where the applicant was team leader
• Personal History Form, using UN Women Personal History Form (P11)
• Financial proposal: the financial proposal shall specify a lump sum amount breaking down the professional fee for each deliverable.

Annexes

After the selection of the evaluation consultant/firm, the following documents will be appended to the ToR:

• Terms of Reference of the Joint Evaluation of the EmPower Regional Project on Gender, Climate Change & Disaster Risk Reduction
• UNEG Norms and Standards for Evaluation (2016): link
• UNEG Ethical Guidelines for Evaluations (2020): link
• Integrating Human Rights and Gender Equality in Evaluation (2014): link 
• UN Women GERAAS evaluation report quality checklist
• UN Women Evaluation Consultants Agreement Form
• UNEG Norms and Standards for Evaluation in the UN System
• UN Women Evaluation Handbook
• UNEP Guidance (to be provided to the selected consultants)

At UN Women, we are committed to creating a diverse and inclusive environment of mutual respect. UN Women recruits, employs, trains, compensates, and promotes regardless of race, religion, color, sex, gender identity, sexual orientation, age, ability, national origin, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, competence, integrity and organizational need.

If you need any reasonable accommodation to support your participation in the recruitment and selection process, please include this information in your application. 

UN Women has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UN Women, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. All selected candidates will be expected to adhere to UN Women’s policies and procedures and the standards of conduct expected of UN Women personnel and will therefore undergo rigorous reference and background checks. (Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.)