Background

 

Introduction

In line with requests expressed by the General Assembly through its Quadrennial Comprehensive Policy Review, resolution 71/243, as well as the Secretary General’s reforms for the repositioning of the United Nations development system to deliver on the 2030 Agenda for Sustainable Development; UNDP, UNFPA, UNICEF and UN Women have committed to working better together, with a view to achieving greater coherence in support of results. This commitment is embodied in a Common Chapter of their respective Strategic Plans for 2018-2021.

In response to request from respective executive boards, the evaluation offices of UNDP, UNFPA, UNICEF and UN Women are jointly evaluating the strategies, implementation modalities, and initial results arising from the agreed joint actions in the ‘key collaborative areas’ set out in the common chapter. The four evaluation offices have established an evaluation management group composed of senior evaluators of the 4 offices to lead this joint evaluation, under the oversight of a steering committee composed of evaluation directors. An initial approach note, annexed to this document, was designed proposing a phased approach to the evaluation of the common chapter, which will be composed of 3 distinct but interrelated exercises: A baseline study; an evaluation of accelerator initiatives, and a final evaluation of the common chapter to be presented to Executive Boards of the four agencies in 2020-2021.

The following presents the terms of reference for a senior evaluation consultant position to assist the development of the first phase of the joint evaluation effort and the baseline study of Common Chapter. It has been agreed amongst the four agencies, that UNDP will manage this consultancy.

 

The joint evaluation of the Common Chapter

Following the adoption of respective strategic plans, the Executive Boards of UNDP, UNFPA, UNICEF and UN Women requested information on how the common chapter was being operationalized by the four agencies; and requested the evaluation offices of the four entities to jointly assess the Common Chapter of the Strategic Plan. Evaluation offices initiated plans for jointly evaluating the implementation of the common chapter at their respective annual sessions in 2018.  An Evaluation Management Group (EMG) composed of senior evaluation officers of the four evaluation offices has been established to manage the Common Chapter evaluation. 

The joint evaluation will be conducted in three phases: a baseline study and evaluability assessment (by March 2020); an evaluation of the accelerator initiatives (by September 2020), and; a summative evaluation of the Common Chapter (by March 2021).

The evaluation will follow the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation in the United Nation System, and abide by the UNEG Ethical Guidelines and Code of Conduct.

 

Objective and purpose of this consultancy

The Evaluation Management Group of the joint evaluation of the Common Chapter is looking for a senior evaluation consultant to lead the first phase of the joint evaluation of the common chapter.

The first phase is expected to serve both as a baseline study and an evaluability assessment of the Common Chapter. As a baseline study, it will determine the status of cooperation among the four entities using evidence from the previous Strategic Plan cycle (2014-17), identifying challenges and advantages of the existent coordination modalities. As an evaluability assessment, it will help inform the scope and overall approach of the 2020-21 summative evaluation of the Common Chapter, by better defining the expectations related to the operationalization of the Common Chapter, and further outlining its key threads and the related data availability.

More specifically, the first phase of the evaluation will provide:

  • Information on the status (level and quality) of recent cooperation between/among UNDP, UNFPA, UNICEF, and UN Women in the key areas envisaged in the Common Chapter. The baseline data thus collected will allow the formative evaluation to assess the extent to which the operationalization of the Common Chapter is contributing to better and more coordinated planning, programming, monitoring, reporting and evaluation, and business operation procedures. The study will not collect primary data on the effectiveness of past joint programs and coordination modalities, relying instead on existing reports, monitoring and evaluation outputs;

 

  • Further clarity on the operationalization of the Common Chapter. By positioning the Common Chapter in the broader context of the UNDS reform, the study will facilitate the attribution of results to the implementation of the Common Chapter vis-à-vis other initiatives all aiming to enhance cooperation among the members of the UNCT for the implementation of the 2030 Agenda for Sustainable Development;

 

  • A technical assessment of basic evaluation parameters (e.g. quality of design, data availability, etc.), enabling an evaluation of the process and results of Common Chapter implementation. As part of the assessment, the study will reconstruct a theory of change for the Common Chapter, including results, indicators, risks and assumptions, which will help informing the second and third phase of the evaluation.

Scope

The scope and coverage of the assignment will focus on joint initiatives within the 6 key areas of collaboration, and those initiatives aimed at enhancing the ways the four agencies work together at headquarters, regional and country level. It will include a review of past experiences and lessons from cooperation among the four agencies, drawing from the implementation of the previous strategic plan 2014-2017. The study will consider all initiatives that have been identified as relevant, excluding actions undertaken by individual entities in fulfilling their core mandates. While data collection will focus primarily on initiatives by UNDP, UNFPA, UNICEF and UN Women, the study will frame those in the broader context of inter-agency partnerships for the achievement of the goals of the Common Chapter. The possibility of using secondary data on the collaboration between the four agencies and other United Nations entities for benchmarking purposes is anticipated.

Key Questions, Data Collection and Analysis Methods

The conduct of the baseline study is expected to be guided by, and to answer, the following evaluation questions:

In the previous strategic plan cycle (2014-2017), to what extent have UNDP, UNFPA, UNICEF, and UN Women been cooperating to achieve common results?

  • What has been the scale, nature, and level of integration of programming and operational services among the four entities?
  • What implementation and coordination modalities have the four entities mostly taken advantage of?
  • At what level (Headquarters, regional office, country office) have the four agencies been mostly working together?
  • In which of the key programmatic areas identified by the Common Chapter have the four entities been collaborating the most?
  • What key factors promoted or constrained the cooperation among the four entities?

To what extent do stakeholders share a common understanding and vision for the implementation of the Common Chapter?

  • To what extent has the design of the Common Chapter built on past experiences in working together and collaborative advantage of agencies?
  • What do stakeholders foresee as opportunities and/or challenges in the operationalization of the Common Chapter?
  • To what extent have demands and expectations from the Common Chapter remained relevant in light of the ongoing reform of the UNDS?
  • To what extent is the operationalization of the Common Chapter coordinated with the activities of other United Nations organizations (including the regional commissions), private sector, and civil society?

How have UNDP, UNFPA, UNICEF, and UN Women conceived the operationalization of the Common Chapter?

  • To what extent is the Common Chapter influencing the way the four entities plan, implement and report progress against common results?
  • To what extent is the Common Chapter affecting coordination structures, management practices, and financing mechanisms at Headquarters, regional and country level?
  • To what extent do the initial accelerator initiatives reflect the demand for greater coherence and enhanced efficiency?

How will the summative evaluation of the Common Chapter be able to determine the effectiveness of the Common Chapter’s implementation?

  • To what extent are the Common Chapter results clearly articulated, both at strategic and programmatic level?
  • To what extent are indicators properly pitched to assess the achievement of results?
  • To what extent are adequate monitoring frameworks and processes in place to enable data collection and the assessment of results?
  • What parameters and benchmarks can be derived from past evaluations and lessons learnt to evaluate the effectiveness of the Common Chapter?
  • What methods should be considered by the summative evaluation to allow for triangulation and objectivity of information?
  • What are the key contextual factors, both internal and external to four agencies that have a bearing on the implementation of the Common Chapter and its evaluability?
  • To what extent will the summative evaluation be able to identify the contribution of the Common Chapter to joint results and enhanced coordination, in the context of the ongoing reform the UNDS and UNDAF at country level?

Methodology

The baseline study will follow a mixed-method approach, but mostly relying on qualitative sources of information which will be quantified as relevant. The study will rely primarily on the following sources of information:

  • Desk review of strategic and programmatic documents to define an inventory of existing coordination mechanisms, and new modalities created because of the adoption of the Common Chapter;
  • Desk review of key documents related to the operationalization of the UNDS reform, to frame the Common Chapter implementation in that context;
  • Analysis of data related to joint programs implemented in the period 2014-17;
  • Meta-analysis of evaluations of joint programs and coordination modalities implemented in the period 2014-17;
  • Interviews with UNDP, UNFPA, UNICEF, and UNFPA Senior Management at Headquarters, Regional and – to a lesser extent - Country level;
  • Interviews with key representatives of United Nations entities that have a system-wide coordination mandates, such as the Executive Office of the Secretary-General, the United Nations Development Coordination Office, UNDG, UNDESA, the Joint SDG Fund, etc.;
  • Interviews with a selected sample of Member States representatives; and
  • Web-based survey of regional and (a sample of) country-level programme staff.

In addition, the study will explore opportunities to gather country-level information related to the Common Chapter’s implementation through data collection activities scheduled in the evaluation work-plan of the four evaluation offices.

Duties and Responsibilities

 

Expected tasks, outputs and deliverables

The consultant is expected to conduct the following tasks:

  • Desk review and secondary data collection
  • Organize a workshop to develop the Theory of Change for the Common Chapter with EMG, technical group and key stakeholders, followed by a feedback meeting
  • Develop interview questions and protocols, and conduct interviews with key representatives of UNDP, UNFPA, UN WOMEN and UNICEF in the Agencies New York headquarters and skype interviews with Agencies regional offices (or Country offices if deemed necessary)
  • Design web survey’s questions
  • Incorporate and address comments from Agencies on the baseline study

The consultant, under the supervision of the management group of the evaluation, will develop a report on evaluating the Common Chapter by September 30th, 2019 composed of two sections that include:  

  1. a summative analysis of the current level of cooperation among the four agencies derived from the desk review of existing evidence around joint programs and cooperation modalities, interviews, and survey; and
  2. the evaluability assessment for the next two phases of the joint evaluation, including a reconstructed theory of change for the Common Chapter, key evaluation questions, and detailed methodology.

Competencies

Core Competencies

  • Demonstrates integrity by modelling UN values and ethical standards
  • Acts as a team player and facilitate team work
  • Maintains confidentiality

Functional Competencies

  • Strong research and analytical skills, including ability to collect reliable, valid and accurate information; and quickly digest and analyze a diverse range of information and data collected from varied sources
  • Experience working on human rights, gender equality, with marginalized groups, and areas of the collaboration listed in the common chapter
  • Knowledge of UN system and the SDGs
  • Use of Microsoft Office Word, Excel, PowerPoint
  • Knowledge of software packages available for quantitative and quantitative analysis, including databases, spreadsheets and statistical packages
  • Ability to prioritize tasks; and plan, organize, and control resources, procedures and protocols to achieve specific goals (project management skills).

 

Development and Operational Effectiveness

  • Strong oral and written communication skills
  • Time management and ability to handle a large volume of work
  • Knowledge sharing and lesson learning
  • Manages conflict

Required Skills and Experience

Education

  • Masters’ degree or equivalent in the social sciences or a relevant field

 

Experience

The consultant is expected to undertake the final baseline study report.  The consultant should have the followings:

  • At least 15 years of extensive background in designing and carrying out thematic and project and thematic evaluation (previous experience with UN system wide evaluations highly preferred) 
  • Experience in the areas of UN joint programmes as well as with inter-agency cooperation mechanisms
  • Expertise  with qualitative and quantitative methods of data collection
  • Capability of independently leading and conducting interviews
  • Strong drafting and analytical skills

Duration of the assignment and indicative timeframe

The data collection, analysis and written inputs by the consultant will take place from June 2019 to March 2020 for 50 working days with the majority of work to be performed between June 2019 and November 2019.

Payments will be made as follows:

  • Upon completion of desk review report: 20%
  • Upon analytical report (draft): 50%
  • Upon completion of all tasks and reports and necessary revisions: 30%

As per UNDP Travel guidelines, the standard for air travel authorized by UNDP for individual consultant is economy class and should the consultant choose to arrange travel by him/herself, he/she will receive the travel entitlement at full fare economy class from UNDP for each mission. Actual settlement of travel cost will be based on invoice of ticket purchased and be paid up to the entitlement amount. Daily Subsistence Allowance (DSA) will be paid per nights spent at the place the mission takes place following UNDP DSA standard rates. Travel costs will be settled separately from the consultant fees. 

Evaluation Process

Applicants are reviewed based on required skills and experience stated above and based on the technical/financial evaluation criteria outlined below. Applicants will be evaluated based on cumulative scoring.