Background

TERMS OF REFERENCE

for Individual contract

1. Introduction

United Nations Development Programme (UNDP) conducts outcome evaluations to capture and demonstrate evaluative evidence of UNDP’s contributions to development results at the country level as articulated in the Country Programme Document (CPD). These are independent evaluations carried out within the overall provisions contained in the UNDP Evaluation Policy[1]. In line with the Evaluation Plan of UNDP Timor Leste, an outcome evaluation will be conducted to assess the impact of UNDP’s development assistance in the Practice Area of Democratic Governance. The proposed evaluation will evaluate the relevant country programme outcomes and outputs as stated in the Country Programme Action Plan (CPAP) and the CPD for Timor Leste both covering the period 2015 - 2019.

Under this practice area, the Country Office has been working to support the country to achieve the Sustainable Development Goals (SDGs) and reduce human poverty. UNDP’s on-going Country Programme Action Plan 2015-2019 (CPAP) is positioned within the overarching objective of the Timor Leste-United Nations Development Assistance Framework (UNDAF) 2015-2019, which elaborates on four outcomes and seventeen sub-outcomes to respond to the evolving needs within each sector. The formulation of the UNDAF in turn has been guided by the SDGs and the National Development Priorities which were defined in the Government’s Strategic Development Plan 2011-2030.

The purpose of the outcome evaluation is to:

• Provide substantive direction to the formulation of programme and project strategies
• Support greater UNDP accountability to national stakeholders and partners in Timor Leste
• Serve as a means of quality assurance for UNDP interventions at the country level and,
• Contribute to learning at corporate, regional and country levels.

The outcome evaluation will be conducted in 2018 towards the end of the current programme cycle of 2015-2019 with a view to improving the implementation of the programme and also providing inputs to the preparation of the new UNDP Country Programme starting from 2020 as well as the forthcoming United National Development Assistance Framework scheduled to start in the same year.

1. Background

The subject of this outcome evaluation will be the projects implemented within the framework of the Democratic Governance portfolio as summarized in Annex D. In addition to assessing the overall result and development impact of the programmes in this practice area, this evaluation will also take into consideration the impact of these programs on cross-cutting issues identified in the CPAP such as gender equality, youth and decentralization. The UNDP Timor-Leste governance programme ensures that UNDP’s projects are aligned under the CPD outcomes and that they do provide specialised knowledge in areas of justice, institutional capacity building and anti-corruption. Importantly, UNDAF outcomes are established with consideration of the relevant social, cultural and traditional contexts of Timor-Leste in the implementation of projects.

The UNDAF 2015-2019 elaborates on four outcomes and seventeen sub-outcomes to respond to the evolving needs within each sector, based on the vision of the Government of Timor-Leste and the comparative advantage of the UN System.  UNDAF outcome 4: ‘State institutions are more responsive, inclusive, accountable and decentralized for improved service delivery and realization of rights, particularly of the most excluded groups,’ links effective democratic governance in Timor-Leste as central to development challenges.

The UNDP’s supports democratic governance by enhancing the effectiveness and integrity of governance institutions, systems and processes. Innovative sustainable development is at the forefront of UNDP’s projects. The partnerships forged by the UNDP with relevant democratic governance institutions and ministries enables the process of strengthening legal frameworks and systems. In Timor-Leste, it is important to balance the importance of traditional customs with international standards of good governance. Thus, the governance programme works in partnership with different sectors and civil society organisations to promote specific and sensitive development mechanisms.

The governance programme currently has six ongoing projects:

1. Supporting Effective Service Delivery and Decentralized Public Administration (AC&D):

This project relates to UNDAF outputs 4.2 and 4.3. This project enhances the collaboration between UNDP and the Government of Timor-Leste in establishing legal and institutional frameworks for anti-corruption and does support the basic service delivery in a few municipalities following the decentralization process. The project strengthens anti-corruption efforts by socializing relevant laws and regulations, strengthening the capacity of the Anti-Corruption Commission (KAK) and increase internal oversight of the commission. The support to the decentralized public administration and strengthening of the democratic participation at the municipality level will also increase rural participation and awareness of adequate, good governance. National partners:

• Anti-Corruption Commission
• Ministry of State Administration
• Two pilot municipalities (not confirmed which yet)

The project is funded by the Korean International Cooperation Agency (KOICA).

1. Leveraging Electoral Assistance for Regionalized Nation-Building (LEARN):

This project relates to the UNDAF Outputs 3.3 and 3.4. The LEARN project in Timor-Leste supports the parliaments, constitution making bodies and electoral institutions to perform their core functions. The project aims to improve accountability, civilian participation and representation in the voting process. It promotes peaceful transitions for all electoral systems and processes. In Timor-Leste, there have been three parliamentary elections held in the last decade. The LEARN project has been working on strengthening technical assistance during and before the parliamentary elections and engaged various CSOs in awareness raising, as well as increasing participation of vulnerable groups (women, youth and people with disability). National partners:

• National Commission for Elections (CNE)
• Technical Secretariat for Electoral Administration (STAE)

The project is funded by the Government of Republic of Korea, the Government of Japan and the Korea International Cooperation Agency (KOICA).

1. The Justice System Programme (JSP):

The JSP project relates to UNDAF outcomes 3, 3.1, 3.4 and 4. The Justice Programme in Timor Leste is the longest programme the UNDP has had, started in 2003. As of 2016, the JSP focussed mainly on two areas: capacity development and access to justice. One of the main focuses of the JSP has been increasing the productivity of the inspectoral system at the Office of the Prosecutor General through the assistance of an international advisor. The UNDP also supports access to justice through a ‘mobile court initiative’ in which municipalities far from the formal courts in Dili aiming at increasing the access to justice. National partners:

• The Ministry of Justice (including the Legal and Judicial Training Center)
• The Court of Appeal
• The Office of the Prosecutor-General
• The Public Defender’s Office

The project is funded by the Government of Japan and the Government of Republic of Korea.

1. UNDP Capacity Building Support to Policia National da Timor-Leste (PNTL):

The PNTL project relates to UNDAF output 3.1. The project works to strengthen governance and service delivery in Timor-Leste through the Timor-Leste National Police. UNDP has sought to strengthen management and administrative capacities of the PNTL to provide access to effective justice and protection to the citizens. Peace is essential for the development of Timor-Leste and providing accountable security services is essential for peace. The second phase of the PNTL project runs from 2016- 2018 and primarily focusses on strengthening the administrative capacity and service delivery of PNTL. UNDP has assisted PNTL to provide continuous support to the people of Timor-Leste. National partners:

• Timor Leste National Police (PNTL)

The project is funded by the Australian Federal Police, the Government of Japan and the Government of the United States of America.

1. Support for the Institutional Development of the Authority of the Oe-Cusse Special Administrative Region and Special Economic Zones for Social Market Economy (ZEESM):

The ZEESM project relates to UNDAF outcomes 1 & 2. The UNDP’s support under the Oe-Cusse Project is built around ‘Support to Governance’ and ‘Technical Assistance for Sustainable and Innovative Development’. One component of the project so far targeted the inconsistencies between localized health posts data in remote areas to regional levels of health management. The implementation of the planned activities during the year 2018 has been affected by the early election which took place on the 12th of May 2018. Going forward, UNDP focuses on the implementation of the Agribusiness Program and the Community Based Tourism Development program. National partners:

• Special Administrative Region (SAR) of Oe-Cusse

The project is funded by the Government of Timor Leste.

1. The Infrastructure Development Support Project (IDSP):

This project is in it’s inception phase and is set to start in the end of May 2018. National partners:

• Major Project Secretariat, the Infrastructure Fund

The project is funded by the Government of Timor Leste.

All projects produce quarterly and annual reports to mark their progress and map out their implementation in line with the relevant project and UNDAF outputs.

[1] www.undp.org/eo/documents/Evaluation-Policy.pdf . The ADR will also be conducted in adherence to the Norms and the Standards and the ethical Code of Conduct established by the United Nations Evaluation Group (www.uneval.org).

Duties and Responsibilities

1. Scope of the Evaluation

The outcome evaluation seeks to:

• Review the UNDP Timor Leste Governance Programme with a view to understand its relevance and contribution to national priorities for stock taking and lesson learning, and recommending corrections that may be required for enhancing effectiveness of UNDP’s development assistance; 
• Review the status of the outcome and the key factors that have affected (both positively and negatively, contributing and constraining) the outcome;
• Assess the extent to which UNDP outputs and implementation arrangements have been effective for building capacities of key institutions which implement government schemes and policies (the nature and extent of the contribution of key partners and the role and effectiveness of partnership strategies in the outcome);
• Review and assess the Programme’s partnership with the government bodies, civil society and private sector and international organizations in Programme;
• Review links/joint activities with other UNDP Programmes and UN Agencies and how these have contributed to the achievement of the outcome;
• Provide recommendations for future Country Programme regarding ways in which the UNDP resources can most strategically impact change in capacities of key institutions of the country so that the delivery mechanisms of the Government are better designed, suit their purpose, and that governance systems put inclusion at the centre of Government efforts;
• Through this evaluation UNDP Timor Leste seeks to understand and articulate the key contributions that the Governance programme has made in the programmes on democratic governance processes, a rigorous analysis of the areas of synergy between the various capacity development strategies adopted within the programme and with other practice areas of UNDP Timor Leste and recommendations to strengthen UNDP’s interventions in this critical area of engagement with the Government of Timor Leste.

The outcome evaluation will be conducted in 2018, and will thus, provide strategic direction and inputs to adequately prepare the United Nations Development Assistance Framework (UNDAF) and UNDP Country Programme after 2020. UNDP will incorporate the findings of the evaluation, experiences and lessons learned while preparing the new Country Programme Document. This evaluation is also expected to bring recommendations regarding partnership to inform a resource mobilization strategy and implementation strategies which could help to (re-)design modalities and management arrangements.

1. Outputs/Deliverables of the Evaluation

1. Inception Report: The inception report which details the evaluators understanding of the evaluation and how the evaluation questions will be addressed. This is to ensure that the evaluator and the stakeholders have a shared understanding of the evaluation.  The inception report will include the evaluation matrix summarizing the evaluation design, methodology, evaluation questions, key informants, data sources and collection analysis tools for each data source and the measure by which each question will be evaluated. The inception report is to be discussed with the UNDP Country Office.

1. Draft Governance Outcome Evaluation Report to be put forward during pre-validation workshop (30 -50 pages). The report will be reviewed by all stakeholders to ensure that the evaluation meet quality criteria. The suggested table of content of the report is as follows:

1. Executive summary
2. Introduction (Background and approach/methodology)

3. Development context (Presentation of issues in the country context. Factual)

4. Development results (Presentation of findings based on the evaluation criteria and other cross-cutting issues. Analytical.) 

5. Bottlenecks for and drivers of success (Analytical)

6. Conclusions and Recommendations 
7. Annexes

1. Final Governance Outcome Evaluation Report, integrating feedback voiced during pre-validation workshop 10 days after receiving the draft report.

Competencies

1. Methodology

The proposed evaluation approach is outlined below. Should there be changes to the approach by the evaluation team, they need to be in accordance to the international criteria and professional norms and standards adopted by the UN Evaluation Group, and previously approved by the UNDP. The Outcome Evaluation will be carried out in accordance with UNEG Evaluations Norms and Standards for Evaluation and OECD/DAG Principles.

The evaluation team will visit selected project sites to meet the local stakeholders and beneficiaries including Central, State and district government officials, civil society organizations including resource agencies, local authorities including elected women representatives, women leaders with potential to contest assembly elections, academics and subject experts, community members and volunteers, etc.

The evaluation shall assess the following for each outcome in the 2015-2019 programming cycle in this portfolio:

• Relevance: Are the project and programme outcomes relevant to UNDP’s mandate, to national priorities and to beneficiaries’ needs? (Relevance to UNDP’s country programme)
• Effectiveness: Have the intended impacts been achieved or are they expected to be achieved? Do different outcome definitions feed into each other and is there a synergy in between? Is the outcome achieved or has progress been made to achieve? Has UNDP made significant contributions in terms of strategic outputs?
• Efficiency: To what extent do the outcomes derive from efficient use of resources? And to what extent UNDP has contributed to the outcomes versus that of its partners?
• Degree of Change: What are the positive or negative, intended or unintended changes brought about by UNDP’s intervention in these outcomes?
• Sustainability: Will benefits/activities continue after the programme cycle?

All evaluation questions should include an assessment of the extent to which programme design, implementation and monitoring have taken the following cross cutting issues into consideration:

• Human rights
• Gender Equality
• Capacity development
• Institutional strengthening
• Innovation or added value to national development
• South-South Cooperation

The outcome evaluation will include the following key activities:  

• Evaluation design and workplan (to be shared with UNDP Timor Leste before start of the evaluation)
• Desk review of existing documents
• Briefing with UNDP Timor Leste
• Field visits
• Interviews with partners
• Drafting of the evaluation report
• Debriefing with UNDP Timor Leste
• Finalization of the evaluation report (incorporating comments received on first draft)

Though the evaluation methodology to be used will be finalized in consultation with the UNDP Timor Leste Country Office, the following elements should be taken into account for the gathering and analysis of data:

• Pre-assessment of data availability
• Desk review of relevant documents including Country Programme Document (CPD), Country Programme Action Plan (CPAP), Project Documents/ Briefs, project evaluations (interim and final), reports of relevant flagship projects, etc.
• Discussions with the Senior Management and programme staff of UNDP Timor Leste
• Presentation of an inception report and discussion of the content with UNDP management and partners
• Interviews: with key partners and stakeholders both at central and field levels. Drawing on specific check listed questions supporting the DAC criteria
• Focus group discussions: within UNDP and external parties both at central and field levels. Gaining consensus on key issues.
• Participation and providing guidance to an Outcome Board Meeting of the Democratic Governance outcome
• Field visits to select key projects, if necessary
• Regular consultation meetings with the UNDP staff, project staff and senior management as appropriate

1. Implementation Arrangements

This evaluation exercise will be undertaken by an Evaluation Expert. The Evaluation Expert will be reporting to the Country Director of UNDP Timor Leste, with delegated authority to the Governance Programme Analyst. The Governance Programme Analyst will arrange the introductory meetings within UNDP and will establish the first contacts with the government partners and project staff. The expert will then set up his/her own meetings and conduct his/her own methodology upon approval of the methodology submitted in the inception report.

The draft and final reports will be submitted in English. The expert will work home/office-based with presence in UNDP premises in Dili as needed for the desk reviews and will get support (contacts and vehicle) from UNDP in arranging their travel arrangements for the site visits.

The Evaluation Expert shall plan all the resources he/she needs to complete the assignment within their financial proposal to the UNDP. The resources to be used by the expert shall be subject to UNDP approval.

The Evaluation Expert will have the overall responsibility for the conduction of the evaluation exercise as well as quality and timely submission of the final evaluation report to UNDP. S/he will specifically undertake the following tasks:

- Lead and manage the evaluation mission,

- Design the detailed evaluations scope, methodology and approach,

- Conduct the outcome evaluation in accordance with the proposed objective and scope of evaluation

- Draft, communicate and finalize the evaluation report as per the comments from UNDP Timor Leste.

Required Skills and Experience

Required Qualifications

• Minimum Master’s degree in economics, public administration, regional development/planning or any other social sciences related to governance issues especially decentralization and local governance, capacity development and democracy.

• At least 10 years of experience in conducting evaluations, preferably of programmes focused on decentralized governance, accountability, capacity development and access to justice.
• Strong working knowledge of UNDP and its mandate, the civil society and working with government authorities.
• Extensive knowledge of results-based management evaluation, as well as participatory M&E methodologies and approaches.
• Experience in applying SMART (S Specific; M Measurable; A Achievable; R Relevant; T Time-bound) indicators and reconstructing or validating baseline scenarios.
• Minimum 10-15 years of professional experience in the area of development, including democratic governance, poverty reduction, regional development, gender equality and social policies.
• Strong reporting and communication skills.
• Excellent communication skills with various partners including donors.

1. Timeline and Schedule

*Each and every activity to be conducted by the Consultant is subject to UNDP approval. Each step shall be conducted upon approval of the previous step by UNDP.

**# of days to be invested by the Consultant may be subject to change based on the project needs and at the sole discretion of UNDP.

The Evaluation Expert will be logistically and financially responsible for arranging his/her travel to and from relevant project sites and arranging interviews. This will also be included in the proposal including the travel costs to mission sites and daily subsistence allowance (DSA), with explicit information presented with the proposal and the methodology.

This work is expected to take 30 working days over a maximum period of 2 months.

The outcome evaluation will be completed by the end of September 2018, with some draft findings presented by mid-August.

1. Schedule of Payments

The individual consultants shall be paid the consultancy fee in USD under an Individual Contract, upon completion of the following milestones.

• 30% after adoption of the inception report
• 50% after presentation and approval of the draft report
• 20% after the approval of the final report

The consultancy fee instalments will be paid as Lump Sum Amounts inclusive of expenses related to the consultancy. The amount paid to the consultant shall be gross and inclusive of all associated costs such as social security, pension and income tax, etc.

1. Application procedures

Individuals and companies are encouraged to apply. In case of individuals, please apply as a team of two. Interested candidates should apply by presenting the following documents:

1. Letter of Confirmation of Interest and Availability using the UNDP template;
2. Personal CV or P11, indicating all experience from similar evaluations, as well as the contact details (email and telephone number) of the candidate and at least three (3) professional references;
3. In case of company, the profile of company indicating suitability to the TOR above and names and CVs of proposed team members;
4. Brief description of why the individual considers him/herself as the most suitable for the assignment, and a methodology, if applicable, on how they will approach and complete the assignment.

[1] Financial Porposal template https://intranet.undp.org/unit/bom/pso/Support%20documents%20on%20IC%20Guidelines/Template%20for%20Confirmation%20of%20Interest%20and%20Submission%20of%20Financial%20Proposal.docx

[2] P11 template http://www.undp.org/content/dam/undp/library/corporate/Careers/P11_Personal_history_form.doc

1. Submissions will be evaluated in consideration of the Evaluation Criteria as stated below:

Applicants meeting the minimum qualifications mentioned above will be short-listed and asked for price proposal. The selection will be made on quality-cost basis, whereby technical component will carry 70% weight and financial component will carry 30% weight.

ANNEX A

What are Outcome Evaluations?

An outcome evaluation is an evaluation that covers a set of related projects, programmes and strategies intended to bring about a certain outcome. Such evaluations assess how and why outcomes are or are not being achieved in a given country context and the role UNDP has played. They may also help to clarify underlying factors affecting the situation, highlight unintended consequences (positive and negative), recommend actions to improve performance in future programming, and generate lessons learned. These periodic and in-depth assessments use “before and after” monitoring data where available.

Outcome evaluations may fulfil different needs at different times throughout the programming cycle. If conducted early in the cycle, they can supply information about potential impediments; if conducted halfway through the cycle, they can suggest mid-course adjustments; and if conducted at the end, they can contribute to lessons learned that could guide work in the outcome during the next cycle. An outcome evaluation extracts lessons learned, findings and recommendations by assessing the following:

• Progress towards the outcome;
• Factors contributing to the outcome (substantive influences);
• Key UNDP contributions (outputs, including those produced through “soft” assistance) to outcomes;
• The partnership strategy (how UNDP works with its partners).

Results-based Management (RBM) Methodology

Since 1999, UNDP has pursued a major programme of reform and renewal with a central objective: to demonstrate how and where the organization is making a measurable contribution to the elimination of poverty. This effort depends on results-based management (RBM), a methodology in which performance at the level of development goals and outcomes is systematically measured and improved, and resources are strategically managed and put to the best possible use to enhance the organization’s development effectiveness. For UNDP, this shift to a “culture of performance” calls for all programming instruments— including monitoring and evaluation—to be aligned with the RBM methodology.

RBM measures how UNDP contributes towards positive development outcomes. It is a system of measuring performance and adopting effective strategies that allow for maximum impact. By changing its focus from outputs to strategic outcomes, RBM allows UNDP to aim for results that ultimately benefit people’s lives by focusing on outcomes.

Outcomes are defined as the developmental changes through the completion of outputs and the achievement of impact (see Figure 1), and are achieved in partnership with others. Partners are agents or actors with whom UNDP has, or intends to have, a substantive relationship in the pursuit of common outcomes. Partners may include stakeholders, if they are involved in working towards the outcome; beneficiaries of outcome actions; and donors involved in some way with UNDP on the outcome.

  Figure 1

Outcome evaluations move away from the old approach of assessing project results against project objectives towards an assessment of how these results contribute, together with the assistance of partners, to a change in development conditions. Outcomes are influenced by the full range of UNDP activities — projects, programmes, non-project activities and "soft" assistance within and outside of projects. Outcomes are also influenced by the activities of other development actors.

The shift towards measuring outcomes also places greater emphasis on building strategic partnerships - because no single agency or project can produce the desired results on its own.  UNDP evaluates its interventions at the level of outcomes rather than inputs and outputs because this level reveals more about how effective UNDP’s actions are in bringing about real change.

This Evaluation will be conducted in accordance with the principles outlined in the Ethical Guidelines for Evaluation (UNEG 2007) and the evaluators must describe, in the inception report, the procedures they will use to safeguard the rights and confidentiality of their sources (e.g. measures to ensure compliance with legal codes governing, for example, provisions to collect and report data, particularly permissions needed to interview or obtain information about children and young people;  provisions to store and maintain security of collected information; protocols to ensure anonymity/confidentiality.)

ANNEX B

DOCUMENTS FOR REVIEW BY THE EVALUATORS

1- UNDP Corporate Policy Documents

• Handbook on Monitoring and Evaluation for Results
• UNDP Guidelines for Evaluators
• Ethical Code of Conduct for UNDP Evaluations

2- UNDP Timor Leste CO Documents

• Country Programme Document (CPD) of Timor Leste for 2015-2019
• Country Programme Action Plan (CPAP) for Timor Leste for 2015-2019
• United Nations Development Assistance Framework for the Democratic Republic of Timor Leste, UNDAF 2015-2019
• Project evaluation reports
• Relevant Project Documents and Annual Progress/Final Reports
• Other documents and materials related to the outcome to be evaluated including the commissioned reports for policy support, government and donor reports if available
• Republic of Timor Leste, Strategic Development Plan 2011 - 2030

ANNEX C

Ethical Code of Conduct for UNDP Evaluations

 Evaluations of UNDP-supported activities need to be independent, impartial and rigorous.  Each evaluation should clearly contribute to learning and accountability.  Hence evaluators must have personal and professional integrity and be guided by propriety in the conduct of their business.

Evaluators:

Must  present information that is complete and fair in its assessment of strengths and weaknesses so that decisions or actions taken are well founded

Must disclose the full set of evaluation findings along with information on their limitations and have this accessible to all affected by the evaluation with expressed legal rights to receive results.

Should protect the anonymity and confidentiality of individual informants.  They should provide maximum notice, minimize demands on time, and: respect people’s right not to engage.  Evaluators must respect people’s right to provide information in confidence, and must ensure that sensitive information cannot be traced to its source. Evaluators are not expected to evaluate individuals, and must balance an evaluation of management functions with this general principle.

Evaluations sometimes uncover evidence of wrongdoing.  Such cases must be reported discreetly to the appropriate investigative body.  Evaluators should consult with other relevant oversight entities when there is any doubt about if and how issues should be reported.

Should be sensitive to beliefs, manners and customs and act with integrity and honesty in their relations with all stakeholders.  In line with the UN Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender equality.  They should avoid offending the dignity and self-respect of those persons with whom they come in contact in the course of the evaluation.  Knowing that evaluation might negatively affect the interests of some stakeholders, evaluators should conduct the evaluation and communicate its purpose and results in a way that clearly respects the stakeholders’ dignity and self-worth.

Are responsible for their performance and their product(s).  They are responsible for the clear, accurate and fair written and/or oral presentation of study limitations, findings and recommendations.

Should reflect sound accounting procedures and be prudent in using the resources of the evaluation.

For details on the ethics in evaluation, please see UNEG Ethical Guidelines.

Annex D

APPROVAL

This TOR is approved by:

Signature                                                                                            

Name and Designation                                                                 

Date of Signing