Background

A series of reviews conducted over the years of the Universal Immunization Programme (UIP), meetings at national and state levels, Joint Review Missions and Common Review Missions organized by Ministry of Health and Family Welfare (MoHFW) on a regular basis, have noted that whilst UIP performance in India has improved overtime, there still remain a number of key constraints impacting immunization coverage in the country. Further, there are significant inequities in vaccination coverage in different states based on various factors related to individual (gender, birth order), family area (area of residence, wealth, and parental education), demography (religion, caste) and the society (access to health care, community literacy level) characteristics.

These assessments have identified the following key constraints:

• Infrastructure issues including poor infrastructure of vaccine stores and transportation systems; lack of standards for vaccine stores at different levels; insufficient safety devices for temperature monitoring of bulk vaccines stored at GMSDs, state, and regional stores; difficulties in procuring the right quality of cold chain equipment on time with adequate after sale support; repair kits and spares cold chain technicians and inequitable cold chain point (last vaccine storage site) distribution;
• HR issues such as lack of a support unit with experts on cold chain for both the immunization division of MoHFW and at the state level; lack of induction training and a regular educational programme for staff inducted in the Cold Chain Logistics (CCL) system; insufficient institutional training capacity to manage cold chain and logistics at all levels; shortage of trained manpower and relevant job-aids for managing cold chain at all levels (state, division/regional and district levels); and lack of HR with capacity for VLM at all levels (national, GMSDs, state, district and PHCs). The shortage of HR is more acute in the poor performing states and specifically at the field level;
• Vaccine management issues and non-adherence to principles of logistics management at all levels leading to mismatch between procurement, forecasting and utilization;
• Weak capacity at the state level and inadequate HR to generate evidence based communication strategies, and effective BCC campaigns;
• Information dissemination is not timely, and often mixed messages are received by beneficiaries;
• Weak interpersonal communication (IPC) skills among health workers and community mobilizers affecting appropriate communication of messages;
• Poor monitoring and evaluation for data entry, resulting in errors in data entry and inaccurate data;
• Poor monitoring and evaluation results in insufficient data quality and reporting rates. A vast majority of states have wide gaps in reported and evaluated coverage data;
• Inadequate data quality and reporting rates result in poor surveillance of VPDs and AEFIs;
• Insufficient laboratory capacity and limited trained manpower at the district levels to carry out surveillance;
• Limited focus on operational research for immunization and finding locally suitable solutions.

Context:

Although the MoHFW is actively working to improve immunization coverage rates, key constraints in the current system remain and need to be addressed. In order to increase the immunization coverage in India, the Health System Strengthening Project was envisaged by the Ministry of Health and Family Welfare (MoHFW), with funding supported by the GAVI Alliance.

The project focuses on five delivery areas namely institutional capacity, cold chain management, evidence-based policy-making and service delivery as well as increase demand for vaccination under the Universal Immunization Program (UIP).

These measures aim to increase both the quality and quantity of the vaccines delivery and immunization, targeting 12 of the 18 National Rural Health Mission (NRHM) high priority states where full vaccination coverage is currently below 61%, and eventually help India build a vaccine delivery platform for the 21st century.

The lead implementers for the project are UNICEF, WHO, UNDP and JSI under the coordination of the MoHFW.

HSS Project Goal:

The goal of the proposal is to reduce infant mortality by improving the immunization coverage in India, in alignment with the national target set by the twelfth five-year plan.

Objectives of the HSS Project:

There are 5 clear objectives of the proposal; each objective is led by one of the implementation agencies as given below:

• Strengthen cold chain management in poor performing states through improved human resources capacity, institutional strengthening and supporting supervision; Lead Implementer - UNICEF;
• Design and implement an eVIN that will enable real time information on cold chain temperatures and vaccine stocks and flows; Lead Implementer - UNDP;
• Increase demand for RI through a national BCC strategy; Lead Implementer - UNICEF;
• Strengthen the evidence base for improved policy-making (at all levels) on programmatic areas through a well developed and implemented national M&E plan and research framework; Lead Implementer - UNDP;
• Leverage the success of the National Polio Surveillance Project to strengthen RI service delivery in 8 priority states. Lead Implementer - UNDP.

Objectives of the evaluation:

The HSS grant is envisaged by the Ministry of Health and Family Welfare to provide a catalytic effect on the Nation’s UIP. The HSS project in many ways infuses technical capacity, institutional strengthening and making the system more efficient to deliver immunization services to every child and expecting mother in the country.  Therefore, it is imperative that the project delivers on the objective defined.

The project paves way to innovation, effective strategy and efficient delivery of services that has a national impact especially considering India plan to introduce newer and more complex vaccines in the coming future.

This evaluation will assess the results achieved, and the sustainability of achieved results, through Gavi’s HSS support to India, as well as the catalytic impact of this support.

Duties and Responsibilities

OBJECTIVES AND SCOPE OF THE EVALUATION

Objectives:

The objectives of the evaluation, and the intended use of the results, are as follows:

• Inform planning and implementation: To establish / independently validate baseline and inform the methodology and key questions for the mid line evaluation;
• Accountability: To provide independent verification and assessment of the of activities supported by Gavi under HSS;
• Documentation of results: To summarise the key observed and reported results of HSS support to India;
• Documentation of challenges faced: To summarise the challenges faced during implementation;
• Documentation of lessons learned: To provide comprehensive documentation of lessons learned from HSS support to India, particularly highlighting lessons learned from more innovative interventions; and
• Provision of recommendations: To provide actionable recommendations for any course correction / measures that should be taken to improve results and catalytic impact for the remaining period of HSS2 implementation.

Scope:

There will be two phases to the evaluation of Gavi’s HSS2 grant:

• First, an initial (baseline evaluation) conducted at the outset of HSS2 implementation to establish / independently validate baselines, respond to certain evaluation questions and to inform the methodology and key questions for the second phase;
• A follow up (midline) evaluation to be conducted at a later stage to independently assess progress, implementation, results and challenges/lessons learned.

Considering some activities are an extension of those supported under HSS1, certain aspects of data collection and assessment should cover both HSS1 and HSS2 activities (see Annex 1 for details of specific / priority activities and interventions supported through HSS).

The evaluation should consider and build upon both the ongoing techno-economic eVIN evaluation, and the ANMOL feasibility assessment and any other relevant studies and evaluations.

EVALUATION QUESTIONS

The evaluation of Gavi’s HSS2 grant (including the baseline and midline evaluation) should seek to respond to the following key questions.

DESIGN AND RESULTS

• To what extent have the activities (per Annex 1) been implemented as planned?
  • What are the main reasons explaining any deviance in implementation?
• To what extent were changes made to the design and implementation of activities for HSS2, compared to the design and implementation of the same activities for HSS1?
  • To what extent were these changes based on evidence?
• To what extent have these activities been implemented in an effective and efficient way?
• To what extent has HSS2 support achieved or is on track to achieve its objectives (outcomes/results and impact) as planned?
  • What are the factors explaining these results?
  • To what extent are reported results valid?
• To what extent did HSS1 support contribute to,and is HSS2 on track to contribute to, improvements in coverage and equity?
• What have been the lessons learned from HSS1, and how have these been applied with the intent to improve results for HSS2?
  • Specific consideration should be made to coverage and equity goals

SUSTAINABILITY

• To what extent are HSS supported activities likely to be sustained (programmatically and financially), and scaled up, post Gavi transition?
  • To what extent are the results sustainable?
• What can already be quantified with respect to HSS1 or HSS2 supported activities that have already been transitioned to or are cost-shared with the central/state governments, as an indication of sustainability?

CATALYTIC IMPACT

• To what extent was HSS1 support to India catalytic? What have been the lessons learned?
• To what extent is HSS2 support on track to be catalytic?
• What HSS1 lessons learned were applied to improve implementation of catalytic support for HSS2? 

LESSONS LEARNED

• What are the key lessons learned that can inform improvements to remaining HSS implementation in India, and elsewhere?
  • What are the major strengths and weaknesses of Gavi’s HSS support to India?
  • What have been the unintended (positive and negative) consequences of HSS support for GoI, Alliance partners, Gavi Secretariat?

Structure of the MTR Team:

The MTR Team will have following three members:

• Lead Consultant with Public Health background, in charge of conducting the Baseline & Mid Term Review and leading a group of consultants;
• Consultant with Health Communications background, leading on the communication related performance and supporting the Lead Consultant in conducting the Mid Term Review;
• Associate with analysis and writing skills, to assist the Mid Term Review team.

The responsibilities of the evaluation Team will largely evolve around the following:

METHODOLOGY

In order to respond to the above questions and provide a high-quality report, we recommend the development of an evaluation framework with fit-for-purpose indicators, methods and approaches. The evaluation approach should take into consideration baseline, midline and end

The methodology should include, at a minimum:

• A comprehensive desk review that analyses and builds upon the following:
  • Both HSS1 and HSS2 grant proposal documents;
  • The mid-term review of India’s HSS1 grant;
  • The end-term review of India’s HSS1 grant;
  • Quarterly reporting provided to the Gavi Secretariat;
  • Reporting on agreed results metrics through the grant performance framework (GPF);
  • Related evaluations / studies (for example, work led by UNICEF to draw lessons on SBCC activities, study on ANMOL looking at health workforce burden, eVIN evaluation, ANMOL feasibility assessment);
  • Other relevant immunisation and health monitoring data (for example, data and reporting available through WHO NPSP, Immunisation dashboard, surveys etc.).

• A series of field observations to independently validate and assess the quality of implementation of highlighted interventions. The final selections of districts to be visited should be discussed and agreed upon by the evaluation steering committee, and should be selected from selected States where most HSS interventions are being implemented. The states to be included are as follows:
• Uttar Pradesh;
• Bihar;
• Madhya Pradesh;
• Rajasthan.

(At least 2 to 3 districts will need to be selected from each state. Total of 10 to 12 districts)

Annex 1 – Suggested key activities for evaluation

Table 1. HSS 2 Key Activities (continuation from HSS1 versus new)

Deliverables:

• Inception Report, including proposed methodology and detail plan;
• Draft Baseline & Mid-term evaluation Reports, including final structure of the report and key findings and recommendation to MoHFW and the implementing partners;
• Final Baseline & Mid-term evaluation Report;
• Power Point Presentation with notes.
• Lead Consultant will be responsible for drafting and finalization of ToR for other consultant and associate and participate in their selection process.

Hiring and Reporting:

• The Lead Consultant will be overall responsible to supervise and orient the team with regards to all aspects of the evaluation process, data collection and analysis.
• The consultant will directly report to the MoHFW and the GAVI Secretariat based at MoHFW for all the technical issues and to UNDP for all the administrative issues.

Competencies

• Excellent communication, interpersonal and liaison skills with government, inter-governmental agencies, UN system and international and regional development partners, academic institutions, civil society organizations, and other relevant institutions;
• Demonstrated capacity to write clear and well-constructed reports, training manuals, process documents; and
• Promote the highest standards of ethics and integrity;
• Strong human rights and gender focus;
• Support creativity and innovation;
• Help create an enabling environment for open communication.

Required Skills and Experience

Education:

• Advanced degree in Public Health, Public Policy, Economics, or other related fields. 

Experience:

• Minimum of 15 years of proven experience in program reviews / evaluations / assessments in public health nationally and multi state, in special relation to immunization will be an asset;
• Clear knowledge of the India’s public health structure and service delivery and demand generation mechanisms in the public health sector;
• Clear knowledge and understanding of India’s Universal Immunization Programme (UIP) and implementation mechanism at National and State levels;
• Understanding issues of public health on issues availability, accessibility and other nuances;
• Previous experience of working for/partnering with Government of India, Ministry of Health and Family Welfare / Department of Health and Family Welare and other ministries at the national and state level;
• Demonstrated and proven experience managing a team of experts to address the various needs of this assessment.

Language:

• Written and oral proficiency in English is required.

Travel:

• The position will require minimum 10 days of travel to the designated states.

Duty Station:

• The Consultant is required to be based at New Delhi and report regularly to the GAVI Secretariat based at MoHFW;
• Relocation expenses, if any, would be borne by the successful Consultant.

Financial Matters:

• Payments will be made at the end of the consultancy for the actual days worked, on satisfactory completion of activities, as agreed with the GAVI Secretariat;
• UNDP will cover the cost of consultancy as per the per day consultancy rates;
• UNDP will organize the preapproved official travel;
• Any other travel related claims will be reimbursed as per actuals;
• Any kind of miscellaneous charges i.e. internet, phone etc. would not be reimbursed.

Documents to be submitted:

• CV, indicating all experience from similar projects and at least three (3) professional references.

A brief technical proposal (not more than 3 pages) that includes:

• One page on your individual expertise and experience in special relationship to the terms of reference;
• One to two pages on proposed activities in response to the evaluation responsibilities, methodology and time lines (not exceeding four months);
• Sample of final reports of similar related evaluations/assessments;
• A Financial proposal indicating all-inclusive per day consultancy fee (in INR);
• Declaration for no conflicts of interest i.e. no involvement in implementing any of the activities that are being evaluated.

Evaluation Criteria:

The award of the contract shall be made to the individual consultant whose offer has been evaluated and determined as:

• Responsive - Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

Technical Criteria weight – 70 Marks; Only candidates obtaining a minimum of 49 points (70% of the total technical points) would be considered for the Financial Evaluation.

• Technical Proposal: 35 Marks;
• Relevant Experience:25 Marks;
• Qualification:10 Marks;

Financial Criteria weight – 30 Marks.

Notes:

• Individuals working with institutions may also apply, contract would be issued in the name of institution for the specific services of individual;
• CV, Technical proposal and Financial proposal should be clubbed in one file for uploading as the system has provision for uploading one document only;
• The consultants must bring his/her own computing equipment.

For any clarifications please write to: ic.india@undp.org