UNDP Jobs - 73823- International Consultant for the terminal evaluati

International Consultant for the terminal evaluation of UNDP/GEF Project “Reducing GHG emissions from road transport in Russia’s medium-sized cities”

Location :

Home-based with 1 mission to Russia, RUSSIAN FEDERATION

Type of Contract :

Individual Contract

Starting Date :

01-Sep-2017

Application Deadline :

27-Aug-17 (Midnight New York, USA)

Post Level :

International Consultant

Duration of Initial Contract :

September - December 2017

Languages Required :

English

Expected Duration of Assignment :

25 working days: 15 w/d home-based, 6 w/d on mission in Moscow, 2 w/d in Kaliningrad, 2 w/d in Kazan

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence. UNDP does not tolerate sexual exploitation and abuse, any kind of harassment, including sexual harassment, and discrimination. All selected candidates will, therefore, undergo rigorous reference and background checks.

Background

The project aims to reduce GHG emissions through improved planning and management of the public transport based on efficient monitoring systems and promotion of eco-friendly vehicles. The main outcomes of the project are pilot activities in 5 cities of Russia (Kazan, Kaliningrad, Irkutsk, Krasnoyarsk, Rostov-on-Don), supportive federal policies and legislative framework for sustainable transport in the Russian Federation. Also, the project disseminates and helps to replicate successes of the pilot sites, where it supports development and implementation of integrated urban transport strategies. Those include comprehensive travel demand surveys and traffic management plans, regulations for integrated urban planning, enhanced public transport systems, and promoting innovative transport solutions such as public rapid transit and non-motorized transport modes.

The project implementation started in 2013 and the expected project’s closing date is 31 December 2017.

Duties and Responsibilities

Objective and scope

This terminal evaluation will be conducted in accordance with the rules and procedures established by UNDP and GEF as reflected in the UNDP evaluation guidance for GEF financed projects. The objectives of the terminal evaluation are to assess the achievement of project results, and to draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming.

The project mid-term evaluation (MTE) took place in late 2015 (final report submitted in early 2016). The final evaluation should assess the extent to which the recommendations of the mid-term review have been taken into account by the project.

The terminal evaluation will explore in detail five major criteria:

Relevance: the extent to which the activity is suited to local and national development priorities and organizational policies, including changes over time;
Effectiveness: the extent to which an objective has been achieved or how likely it is to be achieved;
Efficiency: the extent to which results have been delivered with the least costly resources possible;
Results: the positive and negative, and foreseen and unforeseen, changes to and effects produced by a development intervention. In GEF terms, results include direct project outputs, short- to medium-term outcomes, and longer-term impacts including global environmental benefits, replication effects and other local effects;
Sustainability: the likely ability of an intervention to continue to deliver benefits for an extended period after completion. Projects need to be environmentally as well as financially and socially sustainable.

Evaluation approach and method

An overall approach and method for conducting project terminal evaluations of UNDP supported GEF financed projects has developed over time. The criteria of relevance, effectiveness, efficiency, sustainability, and impact, as defined and explained in the UNDP guidance for conducting terminal evaluations of UNDP-supported, GEF-financed projects are to be used for evaluation. A set of questions covering each of these criteria have been drafted and are to be amended, completed and submitted with the matrix as part of an evaluation inception report, and included as an annex to the final report. The evaluation must provide evidence-based information that is credible, reliable and useful. The terminal evaluation should include a mixed methodology of document review, interviews, and observations from project site visits.

The evaluation team is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, in particular the GEF operational focal point, National Project Director, UNDP Project Support Office, project team, UNDP GEF Technical Adviser based in Istanbul and other key stakeholders. Interviews will be held with the following organizations at a minimum: UNDP Istanbul Regional Hub, UNDP-Russia Projects Support Office, Ministry of Transport of the Russian Federation, administrations of pilot cities (Kazan, Kaliningrad), Federal State Institution “Research Center for Complex Transport Projects of the Ministry of Transport of the Russian Federation”, OJSC “Scientific and Research institute of motor transport”, Moscow Automobile and Road Construction University (MADI), Institute for Transport Economics and Transport Policy Studies of the National Research University "Higher School of Economics", municipal institution " Organizer of passenger transportations" (the largest bus public transport operator in Kazan), administrations of pilot replication cities. The evaluation team should also speak with the key international and national consultants of the project including the international consultant who designed and wrote the project document, the project mid-term evaluator and the project international CTA.

A list of persons and organizations for interviews will be proposed by the project team and should be agreed upon at least 1 week prior to the mission to the Russian Federation. The international evaluator can request additional meetings/interviews if required. UNDP and NIM partner should be informed of additionally requested interviews/meetings and the dialogue with the evaluated party should be handled in an inclusive and transparent manner.

The international evaluator will review all relevant sources of information, such as project document, project reports – including annual APR/PIR, project budget revisions, MTR and progress reports, GEF focal area tracking tools, project files, national strategic and legal documents, reports of contractors and sub-contractors, reports of the international CTA, reports of the national consultant, and any other materials that the evaluator considers useful for this evidence-based assessment.

Evaluation criteria & ratings

Assessment of project performance will be carried out against expectations set out in the Project Logical Framework/Results Framework, which provides performance and impact indicators for project implementation along with their corresponding means of verification. The evaluation will at a minimum cover the criteria of: relevance, effectiveness, efficiency, sustainability and impact. Ratings must be provided on the following performance criteria:

Monitoring and Evaluation (M&E design at entry, M&E plan implementation, overall quality of M&E);
IA&EA Execution (Quality of UNDP implementation, quality of execution - executing agency, overall quality of implementation/execution);
Assessment of Outcomes (Relevance, effectiveness, efficiency, overall project outcome rating);
Sustainability (Financial resources, socio-political, institutional framework and governance, environmental, overall likelihood of sustainability).

The completed table must be included in the evaluation executive summary.

Project finance/co-finance

The evaluation will assess the key financial aspects of the project, including the extent of co-financing planned and realized. Project cost and funding data will be required, including annual expenditures. Variances between planned and actual expenditures will need to be assessed and explained. Results from recent financial audits, as available, should be taken into consideration. The evaluator will receive assistance from the Project Support Office and project team to obtain financial data in order to complete the co-financing table, which will be included in the terminal evaluation report.

Impact

The evaluator will assess the extent to which the project is achieving impacts or progressing towards the achievement of impacts. Key findings that should be brought out in the evaluations include whether the project has demonstrated:

verifiable improvements in ecological status;
verifiable reductions in stress on ecological systems;
demonstrated progress towards these impact achievements.

Conclusions, recommendations & lessons

The evaluation report must include a chapter providing a set of conclusions, recommendations and lessons learned. Conclusions should build on findings and be based on evidence. Recommendations should be prioritized, specific, relevant, and targeted, with suggested implementers of the recommendations. Lessons should have wider applicability to other initiatives across the region, the area of intervention, and for the future.

Evaluation timeframe

The total duration of the evaluation will be 25 working days during the calendar period from 01 September till 31 December 2017. The following tentative timetable is recommended for the evaluation; however, the final schedule will be agreed upon in the beginning of the assignment:

Preparation - 3 days in September 2017;
Evaluation Mission - 10 w/days in late September – early October 2017;
Draft Evaluation Report - 10 days, completed by mid of November 2017;
Final Report - 2 days, completed by the end of November 2017/early December 2017

Evaluation deliverables

The international evaluator is expected to deliver the following:

Inception report – the evaluator provides clarifications on timing and method which includes a clear statement of the objectives of the evaluation and the main aspects to be evaluated no later than 1 week before the evaluation mission and submits the report to the UNDP PSO;
Presentation - initial findings at the end of the evaluation mission presented to the project management, UNDP PSO and UNDP Regional Technical Advisor. Approval of the preliminary draft report with the project team and NIM implementing partner;
Draft evaluation report – with included overall rating with annexes within 3 weeks of the evaluation mission sent to UNDP PSO, reviewed by RTA, PSO and project team;
Final report - revised report within 1 week of receiving UNDP comments on draft sent to PSO for uploading to UNDP ERC. When submitting the final evaluation report, the evaluator is required also to provide an 'audit trail', detailing how all received comments have (and have not) been addressed in the final evaluation report. Approval of the final terminal evaluation report with the project team and NIM implementing partner.

Evaluation ethics

Evaluation consultant will be held to the highest ethical standards and is required to sign a Code of Conduct upon acceptance of the assignment. UNDP evaluations are conducted in accordance with the principles outlined in the UNEG 'Ethical Guidelines for Evaluations'.

Payment installments:

10% Following submission and approval by UNDP PSO of a detailed workplan/inception report prior to the evaluation mission;

55% Following submission and approval by UNDP (National Implementing Partner, UNDP-PSO, UNDP RTA) of the 1st draft terminal evaluation report;

35% Following submission and approval by UNDP (National Implementing Partner, UNDP-PSO, UNDP RTA) of the final terminal evaluation report.

Competencies

Corporate Competencies:

Demonstrates integrity by modeling the UN’s values and ethical standards;
Promotes the vision, mission, and strategic goals of UNDP;
Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Functional competencies:

Strong interpersonal skills, communication skills and ability to work in a team;
Ability to plan and organize work, efficiency in meeting commitments, observing deadlines and achieving results;
Openness to change and ability to receive/integrate feedback;
Ability to work under pressure and stressful situations;
Strong analytical, research, reporting and writing abilities.

Required Skills and Experience

Education

University degree (Masters or equivalent) in the field of relevance such as urban planning and development, urban transport planning, civil engineering.

Relevant experience:

Minimum 7 years of professional experience related to the transport sector;
Experience with results-based monitoring and evaluation methodologies;
Experience in UNDP and GEF evaluations is a strong advantage;
Relevant work experience in Europe & CIS region and/or Russian Federation is an advantage;

Language skills

Excellent English, Russian language will be considered as an advantage.

Evaluation procedure

Individual consultants will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal. The award of the contract shall be made to the individual consultant whose offer has been evaluated and determined as:

Responsive, compliant, acceptable;
Having received the highest score out of a pre-determined set of technical and financial criteria specific to the solicitation.

Technical criteria - 70% of total evaluation (max 70 points):

University degree (Masters or equivalent) in the field of relevance such as urban planning and development, urban transport planning, civil engineering (max 5 points);
Minimum 7 years of professional experience related to the transport sector (max 10 points);
Experience with results-based monitoring and evaluation methodologies (max 10 points);
Experience in UNDP and GEF evaluations is a strong advantage (max 15 points);
Relevant work experience in Europe & CIS region and/or Russian Federation is an advantage (max 5 points);
Excellent English, Russian language will be considered as an advantage (max 5 points);
Interview – maximum 20 points.

Financial criteria - 30% of total evaluation (max 30 points). Only candidates passing the 70% threshold for the technical proposal will be considered for the financial evaluation. The candidate with the highest score from technical criteria + financial criteria will be selected with the maximum score possible being 100 points.

Application process