Background

BACKGROUND

The project “Procurement Support Services to the Ministry of Health of Ukraine, Phase II” (2020-2022) funded by the Ministry of Health of Ukraine is a continuation of the Procurement Support Services to the Ministry of Health of Ukraine Project. Back in 2015, UNDP started procuring medicines and medicinal products on behalf of the Ministry of Health of Ukraine, as an emergency response and following a request of the government of Ukraine. This emergency temporary measure was designed to restore supply of medicines to Ukrainian citizens, which had been previously interrupted due to inefficiencies. The respective cost sharing agreement was signed at the end of 2015 and since then UNDP conducted procurement for 111 programmes for the 2015-2019 State Budget funds. In 2020-2021 under the MoH PSS, Phase II Project UNDP was entrusted with procurement of medicines for 14 programmes. The total budget for these budget cycles is USD 90 mln.

The overall goal of the Project is to support the implementation of the health reform in Ukraine through strengthening the medicine procurement capacity and enhancing the efficiency, effectiveness, transparency and accountability of the national healthcare procurement system, which ultimately will improve the quality, accessibility and affordability of health services. The specific objectives of the Project are:

I. To strengthen the capacity of the Ministry of Health of Ukraine to ensure transparency, accountability and effectiveness of the public procurement of medicines and other medical products.

II. To procure medicines and medical products for the national public health programme

III. To support the capacity building of the Ministry of Health in implementation the health procurement reform.

Please find more information about the project in its Project Document at https://open.undp.org/projects/00132175.

The Project follows a human-rights-based approach to programming under which policies, processes and planned activities are anchored in the system of rights and corresponding obligations established by international law and ensures gender-mainstreaming in all its components providing opportunities for equal participation of women and men in capacity building, advocacy and grant activities.

The cross-cutting topic of gender equality is also addressed through this project. Efficient procurement of medicine related to women’s and children's health significantly contributes to promoting gender equality and women’s empowerment as women in Ukraine are primary caregivers for children and especially mothers of children with certain conditions, diseases or disabilities. The project ensures gender mainstreaming by working with women’s and patients’ organizations, preparing gender-sensitive analytical products, ensuring equal involvement of men and women both in expert and training work, promoting gender activities as part of the communication work and other actions.

Additionally, to the health reform challenges, Ukraine continues to have a concentrated HIV epidemic among key populations (including people who use drugs, sex workers, men who have sex with men, transgender people and prisoners). The HIV epidemic is characterized by a growing number of registered HIV cases in most regions of the country. The rapid assessment data of HIV service organizations report indicates growth of risk behaviour (especially in the collective centres for internally displaced persons, losses of prevention networks and increased use of drugs (including injecting). UNDP MoH PSS, Phase II Project contains a component related to HIV/TB response in Ukraine.

Also, in 2018 UNDP began implementation of the Sustainable Health in Procurement Project (SHiPP) in collaboration with Health Care Without Harm (HCWH), and funded by the Swedish International Development Agency, that aims to reduce the harm to people and the environment caused by the manufacture, use and disposal of medical products and by the implementation of health programmes. SHiPP is a four-year project aiming to promote sustainable procurement in the health sector, in the United Nations (UN) Agencies, and in key project countries through the reduction of toxicity of chemicals and materials in health products, the reduction of greenhouse gases in the supply chain and the conservation of resources. Project countries include Guatemala, Moldova, Ukraine, Tanzania, Vietnam, and Zambia. SHiPP is one of the components of MoH PSS, Phase II Project.

As currently MoH PSS, Phase II is in its phasing out stage, it is important to engage the national consultant to support international evaluator to assess the extent to which project objectives were achieved and contribute to future programming, policymaking and overall organizational learning with a focus on lessons learnt and best practice.

2. MAIN OBJECTIVE OF THE ASSIGNMENT

The major objective of the assignment is to provide support to independent international consultants in conducting a final evaluation of MoH PSS, Phase II implementation to assess the extent to which the project objectives were achieved, summarize the key results, lessons learned and best practices, analyse project’s relevance, effectiveness, efficiency, sustainability, and coherence, with a view to contribute to future adaptation, programming, policymaking and overall organizational learning by outlining recommendations for the next phase of UNDP’s health activities.

Duties and Responsibilities

DESCRIPTION OF RESPONSIBILITIES/SCOPE OF WORK

The national consultant with work under the supervision and guidance of the Team Leader and in close cooperation with coordination of international consultant. She/he will assist in preparation for and the scheduling of meetings with project stakeholders, UNDP team, translation of documents and interpretation during the meetings, sourcing and compilation of relevant documents for the evaluation and support the evaluation team in the creation of the deliverables of the evaluation effort as described below.

The key products expected to be produced by the evaluation team is a comprehensive evaluation report (up to maximum 25-30 pages without annexes, single spacing, Myriad Pro font, size 11) with key findings and a maximum of 7 key actionable recommendations. The evaluation report should include, but is not limited to the following components:

• Project and evaluation information details
• Executive summary (four pages maximum)
• Introduction
• Description of the intervention (Development context and project background)
• Evaluation scope and objectives
• Evaluation approach and method
• Data analysis
• Key findings and conclusions
• Recommendations (up to 7 actionable recommendations)
• Lessons learned and best practices for the future (including viable ideas on focus areas and work directions which could be sharpened and further enhanced in the next UNDP health support programme)
• Annexes: TOR, list of field visits and their agendas, list of people interviewed, documents reviewed, interview and focus group questions, etc.

In addition to a final evaluation report, the executive summary on the key findings, lessons learned and best practices and recommendations (no more than 5 pages long) has to be produced by the evaluation team.

The evaluation at a minimum will cover the OECD-DAC evaluation criteria of relevance, effectiveness, efficiency, sustainability and coherence. The evaluation team should also address how the Project applied the human rights-based approach and mainstreamed gender in development efforts. The evaluation should be conducted in accordance with the guidance, rules and procedures established by UNDP and as reflected in the UNDP Evaluation Policy.

Specifically, it will cover (but not be limited to) the following areas and questions:

RELEVANCE

The report will examine the extent to which the project is relevant to the:

• Country context: How relevant are UNDP project’s interventions to the Ukrainian society in general and specifically for the Government’s needs and priorities and the current evolving country context?
• Target groups: To what extent was the project relevant to address the needs of vulnerable groups and gender issues (both at project and stakeholder’s level)? To what extent did the initial theory of change for the project take those groups into consideration?
• Describe if the Project was able to transform/adjust to fast changing political context taking into consideration risks/challenges mitigation strategy.  The Evaluator can emphasize to what extent Project outputs have been achieved with involvement of government partners and have been adopted into national strategies, policies and/or legal codes.
• To what extent has the project contributed to CSO’s engagement, the empowerment of patients’ organizations, mainstreaming gender equality and human rights based principles?
• Was the intervention designed in ways that respond to the needs and priorities of all groups? If so, how?

EFFECTIVENESS

• Assess the overall performance of the Project with reference to its respective projects documents/cost-sharing agreements, strategies, objectives and indicators, and identify key issues and constraints that affected the achievement of the project’s objectives. Did the intervention achieve the project objectives and what were the key results at outcome and output levels? What are the results achieved beyond the logical framework?
• What factors contributed to effectiveness or ineffectiveness?
• In  which  areas  does  the  project  have the greatest achievements? Why and what have  been  the  supporting factors? How can the project build on or expand these achievements?
• What could have been done differently in the project (from a methodology and/or activity, or resource usage point of view) to implement the project more effectively?
• How have stakeholders been involved in project implementation? How effective has the Project been in establishing national ownership?
• To  what  extent  has  the  project  been  appropriately  responsive  to  the  needs  of  the  national  constituents and changing partner priorities?
• To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights?

EFFICIENCY

• To  what  extent  was  the  project  management  structure  as  outlined  in  the  project document efficient in generating the expected results?
• To what extent the Project Has been effective? Is the Project using the most cost-effective options? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve the relevant outputs and outcomes?
• To what extent have project funds and activities been delivered in a timely manner?
• Has the cost and delivery time of the procured medicine and medical products decreased, and if yes for how much? Is the procurement conducted by UNDP more efficient than other procurement conducted by state and private agents?
• Was the project management, coordination and monitoring efficient and appropriate?  Is the monitoring considering gender equality, as well as social inclusion and human rights, environmental protection and climate change (if applicable and where relevant)?
• To what extent has the project ensured value for money?
• Has the Project produced results (outputs and outcomes) within the expected time frame? Was implementation delayed, and, if it was, did that affect cost effectiveness or results?
• Were different resources allocated in ways that considered gender equality? If so, how were they allocated? Was differential resource allocation appropriate?

SUSTAINABILITY

• To what extent are the project results (impact, if any, and outcomes) likely to continue after the project ends? Define the areas, which produced the most sustainable results, and the most promising areas requiring further support.
• To what extent will financial and economic resources be available to sustain the benefits achieved by the project?
• Do  the  legal  frameworks,  policies  and  governance  structures  and  processes  within  which  the  project operates pose risks that may jeopardize sustainability of project benefits?
• Is there sufficient public/stakeholder awareness in support of the Project’s long-term objectives?
• Are there any social or political risks that may jeopardize the sustainability of project results?
• To what extent were capacity development initiatives for partner organizations adequate to ensure sustainability? What could be done to strengthen exit strategies and sustainability? What should be phased out? And is there a best practice sustainability model which can be replicated into a future design?
• To  what  extent  do  mechanisms,  procedures  and  policies  exist  to  allow  primary  stakeholders  to  carry forward the results attained on gender equality, empowerment of women, human rights and human development?
• To what extent do UNDP interventions have well-designed and well-planned exit strategies?
• Identifying possible priority areas of engagement, offer recommendations for the next phase

COHERENCE

• To what extent was the project aligned with the policies and strategies of the Government, the UN 2030 Agenda for Sustainable Development as well as UNDP Strategic Plan, UNDP Country Programme Document/United Nations Partnerships Framework? Is the Project in line with the Ukraine’s Recovery Plan and UNDP Resilience Building and Recovery Programme for Ukraine developed after the onset of the full-scale war?
• Is there a coherence with other donors’ interventions? Is the Project building upon/seeking synergies with existing programmes, projects and strategies in order to maximize impact, efficiently allocate resources and avoid duplications?
• Is the Project building upon/seeking synergies with existing programmes and projects with similar objectives in order to maximize impact, efficiently allocate resources and avoid duplications?
• To what extent does the intervention support national legislation and initiatives that aim to improve gender equality and human rights? What lessons can be learned?

Focus and cross-cutting issues, such as HRBA and gender should be carefully evaluated and be integrated across the evaluation.

The final list of evaluation questions and tools to be proposed by the evaluation team and agreed with UNDP in an Inception Report.

Recommendations on the cost efficiency assessment.

As part of the evaluation exercise, an efficiency assessment on the procurement of medicine and medical products needs to be conducted. This scope of activities foresees:

• Analyse changes in the quantity, prices and delivery time of procured pharmaceuticals, medical supplies and equipment within the state programmes implemented by UNDP for the implementation period of the 2020 and 2021 state budget years.
• Provide baselines for the procurement of medical supplies and equipment for the selected programmes.
• Compare the cost of UNDP procurement of medicines and medical equipment with the same items procured by state or nonstate agents.
• Conduct interviews/meetings with key partners for validating the information and data, specifying the procedures, obstacles and decision made that influenced the efficiency of the costs.

Competencies

1. METHODOLOGY AND PROPOSED ARRANGEMENTS

The scope of the final evaluation will cover all activities undertaken in the framework of the Procurement Support Services to the MOH, Phase II Project.

The evaluation must provide evidence based and transparently obtained information that is credible, reliable and useful. The evaluation will need to use mixed methods and tools to ensure that data gathering and analysis deliver evidence-based qualitative and quantitative information, based on diverse sources: desk studies and literature review, cost-benefit analysis, individual interviews, surveys and direct observation. This approach will not only enable the evaluation to assess causality through quantitative means but also to provide reasons for why certain results were achieved or not and to triangulate information for higher reliability of findings. The national consultant is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, CSOs, international partner organisations, UNDP Country Office and Project team.

An evaluation of Project efficiency will be carried out against targets set out in the Project Logical Framework/ Results Framework, which provides performance and impact indicators for Project implementation along with their corresponding means of verification. It is suggested that the evaluation should use a mixed method approach – collecting and analyzing both qualitative and quantitative data using multiple sources in order to draw valid and evidence-based findings and conclusions and practical recommendations. All data provided in the report should be disaggregated by sex and other social variables, where possible.

The evaluation team will also identify lessons learnt and best practices from the Project which could be applied to future and other on-going UNDP interventions.

The conclusions should be comprehensive and balanced, and highlight the strengths, weaknesses, challenges and significant outcomes and outputs of the Project. They should be well substantiated by the evidence and logically connected to the final evaluation findings. They should respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to Project beneficiaries, UNDP and MoH PSS, Phase II.

The final evaluation methodology and approach to be discussed and agreed with UNDP shall include, as a minimum, the following elements / sources of information:

• Desk review of the documents listed below ( but not limited to):

a) The original project documents, databases monitoring reports, action plans, M&E frameworks, assessments, and financial documents (such as the cost-sharing agreement with MoH);

b) Notes from meetings involved in the projects (such as board meeting minutes);

c) Other project-related material produced by the projects (such as datasets, publications, audio-visual materials and consultancies reports).

• Cost efficiency assessment of the procured medicine and medical products for nosologies within 2020-2021 procurement cycle.
• Interviews with the relevant UNDP Country Office and the Project’s management and staff, MoH and the various national and sub-regional authorities dealing with the Project activities as necessary, to provide in-depth briefing on the interventions, its results, context of partnerships with different stakeholders etc. as well as vision for future.
• Interviews and/or focus groups discussions with partners and beneficiaries. Partners and beneficiaries can be divided into four distinct groups:
  • Patient organizations and other civil society organizations and associations;
  • Government institutions (including but not limited to MoH, logistics’ state entities, State expert centre, others);
  • International development actors active in the field of intervention (EU, USAID, UNICEF, WHO etc).
  • Interviews with other key informants/experts.

For each of these key informant interviews, the national consultant is encouraged to develop and present his/her ideas for the content and format of the interview forms (e.g. interview guides defining the structure of future interviews and key proposed questions to be asked) that will be applied to capture the information required, as well as the method to be used in administering them and tabulating the results.

1. DELIVERABLES AND TIMEFRAME OF THE EVALUATION PROCESS:

The Evaluator should provide the following deliverables:

6. IMPLEMENTATIONS arrangements

The detailed schedule of the evaluation and the length of the assignment will be discussed with the evaluator prior to the assignment. The duration of the assignment is 30 working days (December 2022 – March 2023).

A team of three independent consultants will conduct the evaluation - one international team leader (with experience and practice of participation in projects and evaluations in other regions of the world and knowledge of health governance), one international expert (with expertise in public procurement), supported by national evaluation assistant.

Evaluation manager will review and approve inception reports including evaluation questions and methodology, review and comment on evaluation report, circulate draft evaluation report, collect and consolidate comments and share with the Evaluator for finalization of the evaluation report. The satisfactory completion of each of the deliverables shall be subject to the endorsement of the UNDP Evaluation Manager.

The evaluation team will be supported by the Evaluation Focal Team (EFT) comprising of representatives of UNDP Ukraine (Health Programme Specialist, UNDP Democratic Governance Analyst, and relevant project staff). The EFT will assist in providing the available documentation for the analysis and research, setting up the meetings with partners and external actors connecting the evaluation team with the regional partners and key stakeholders, arranging field visits, identifying key partners for interviews.

The evaluation team will inform UNDP of any problems, issues or delays arising during the implementation of the assignment and take necessary steps to address them.

The key product expected is a comprehensive evaluation report (with parameters indicated above). The report must be as free as possible of technical jargon in order to ensure accessibility to its wide and diverse audience. The report should be in line with UNDP’s principles of gender-responsive communication and should be prepared in English.

All reports and results are to be submitted to the UNDP in electronic form (*.docx, *.xlsx, *.pptx, and *.pdf or other formats accepted by UNDP).

1. EVALUATION ETHICS, IMPARTIALITY AND CONFIDENTIALITY

This evaluations will be conducted in accordance with the principles outlined in the United Nations Evaluation Group (UNEG) 'Ethical Guidelines for Evaluations'.

The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

Evaluation consultants will be held to the highest ethical standards and are required to sign a pledge of ethical conduct upon acceptance of the assignment.

The evaluators selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

The consultants must also agree to hold in trust and confidence any information or documents (“confidential information”) disclosed to them or discovered by them or prepared by them in the course of or as a result of the evaluation and agree that it shall be only used for the purposes of this evaluation and shall not be disclosed to any party without UNDP approval.

Required Skills and Experience

1. EXPERIENCE AND QUALIFICATIONS REQUIREMENTS

• University degree in Monitoring and Evaluation, Public Administration, Health, Economics, Public Health, Data analysis, Public Procurement, Social Sciences or related fields;
• At least 3 years of relevant experience including administrative, logistics or project support at national or international level;
• At least 1 year of experience in coordinating communication with different stakeholders, such as government authorities, non-governmental stakeholders;
• At least 1 year of experience in monitoring and evaluation, information gathering, research methodology or data analysis;
• Excellent knowledge of Ukrainian and/or Russian, as well as fluency in spoken English and good command of written English.

1. DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE PROPOSALS

Applicants shall submit the following documents:

1. FINANCIAL PROPOSAL

Lump sum contract

The financial proposal shall specify a total lump sum amount in USD, and payment terms around specific and measurable (qualitative and quantitative) deliverables (i.e. whether payments fall in instalments or upon completion of the entire contract). Payments are based upon output, i.e. upon delivery of the services specified in the TOR. In order to assist the requesting unit in the comparison of financial proposals, the financial proposal will include a breakdown of this lump sum amount (including travel, per diems, and number of anticipated working days).

Travel costs

All envisaged travel costs will be paid separately according to UNDP rules and procedures and should not be included in the financial proposal. This includes all travel to join duty station/repatriation travel. In general, UNDP should not accept travel costs exceeding those of an economy class ticket. Should the Individual Consultant wish to travel on a higher class he/she should do so using their own resources. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and Individual Consultant, prior to travel and will be reimbursed.

1. EVALUATION CRITERIA

• University degree in Monitoring and Evaluation, Public Administration, Health, Economics, Public Health, Data analysis, Public Procurement, Social Sciences or related fields – 12 points max

• 12 pts – Master’s degree;
• 10 pts – Bachelor’s degree;

• At least 3 years of relevant experience including administrative, logistics or project support at national or international level – 15 points max

• 15 pts – 6+ years;
• 12 pts – 4-5 years;
• 10 pts – 3 years;

• At least 1 year of experience in coordinating communication with different stakeholders, such as government authorities, non-governmental stakeholders – 19 points max

• 19 pts – more than 1 year  
• 14 pts – 1 year

• At least 1 year of experience in monitoring and evaluation, information gathering, research methodology or data analysis – 19 points max

• 19 pts – more than 1 year
• 15 pts – 1 year

• Excellent knowledge of Ukrainian and/or Russian, as well as fluency in spoken English and good command of written English – 5 points max

• 5 pts – excellent knowledge of Ukrainian/Russian and English;

Maximum available technical score - 70 points.

1. Evaluation method

Cumulative analysis

Contract award shall be made to the incumbent whose offer has been evaluated and determined as:

a) responsive/compliant/acceptable, and

b) having received the cumulative highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

* Technical Criteria weight: 70%

* Financial Criteria weight: 30%

Only candidates obtaining a minimum 70% from the maximum available technical score (49 points) would be considered for the Financial Evaluation

The maximum number of points assigned to the financial proposal is allocated to the lowest price proposal and will equal to 30. All other price proposals will be evaluated and assigned points, as per below formula:

30 points [max points available for financial part] x [lowest of all evaluated offered prices among responsive offers] / [evaluated price].

The proposal obtaining the overall cumulatively highest score after adding the score of the technical proposal and the financial proposal will be considered as the most compliant offer and will be awarded a contract.