Historique

A. Project Title:

Technical Support to the Department of Panchayati Raj, Government of Odisha to fast-track construction of Rural Houses

B. Background:

The Government of Odisha aimed to construct more than 1.5 million houses for rural poor households by 2022. At the request of the Government of Odisha, UNDP carried out a diagnostic study in the State to understand the challenges and suggest a way forward. Various challenges identified need to be addressed to ensure that the housing provided will be durable, functional, aesthetic and disaster resilient as desired by the State Government. In addition to the allocation of financial resources for construction, beneficiaries need improved access to building materials, context-specific technical advice, design options, and financial services to meet their housing needs. The project was designed to leverage UNDP’s work on rural housing with the Department of Panchayati Raj, Government of Odisha to develop an institutional mechanism that could provide technical and management support and undertake some capacity building and documentation activities. 

Through the project UNDP technical team aimed to facilitate and support the various rural housing schemes of the government,  to ensure completion of 1.45 million poor households in rural areas of the State  through an integrated, context-relevant, affordable, and disaster-resilient housing approach over the period of 2017 to 2021. The specific results of the programme were

• Project Management Unit established at State, district and block levels with 696 personnel deployed for timely and quality construction of houses 

• 150 Master Trainers and 150 masons trained for enhanced quality of houses constructed

• Information, Education, Communication (IEC) materials developed for effective mobilization of project beneficiaries (5 posters, 5 pamphlets, 3 audio jingles, 1 street plays, 10 success stories, 4 video films)

C. Evaluation Purpose:

a. Purpose:

The purpose of the final evaluation is to assess achievements to date, document lessons learned,  understand the degree of gender mainstreaming within the scope of this project, and provide recommendations to UNDP and its partners for any future engagement.

b. Tiimeline:

The final project evaluation is proposed to be conducted in August 2022. The timeline has been agreed with the project steering committee. 

c. Utilization:

The primary users of the evaluation results will be UNDP, but the evaluation results will equally be useful to relevant GoO.

In addition, the evaluation aims at critically reviewing and identifying what has worked well in the    project, what challenges have been faced, what lessons can be learned to improve future rural housing programming. The evaluation will also generate knowledge for wider uses, assess the scope for scaling up in other parts of the country, and serve as a quality assurance tool for both upward and downward accountability. UNDP will take in consideration all useful findings, conclusions, and recommendations from the evaluation, prepare a systematic management response for each recommendation, and implement follow-up actions as per UNDP Evaluation Resource Center guidance/policies.

In the view of the above, UNDP is seeking for one Senior National Consultant to conduct the final evaluation of Rural Housing. S/he will work in collaboration with Junior National Consultant. The evaluation will work under the overall supervision of the Chief, Strategic Partnership and State Outreach and day to day collaboration with the Project Manager Rural Housing. 

D. Objectives of the assignment:

The final evaluation will focus on measuring development results generated by the Rural Housing Project, based on the scope and criteria included in this term of reference. The unit of analysis or object of study for this evaluation is the rural housing, understood to be the set of components, outcomes, outputs, activities, and inputs which are described in the Rural Housing Project document and M&E Framework.

The objectives of this evaluation are to:

• Assess to what extent rural housing project has contributed to address the needs and problems identified during programme design.

• Assess how effectively rural housing project has achieved its stated development objective and purpose.

• Measure how efficiently the rural housing outcomes and outputs have progressed in attaining the development objective and purpose of the project.

• Assess both negative and positive factors that have facilitated or hampered progress in achieving the project outcomes, including external factors/environment, weakness in design, management, and resource allocation.

• Through qualitative methods including participatory approaches explore the impact of the project on women participants at individual, family and community level. Identify and document substantive lessons learned, good practices and opportunities for scaling the project in future rural housing projects in India

• Provide forward looking programmatic recommendations for the rural housing project.

The evaluation will focus on five key evaluation criteria: relevance, efficiency, effectiveness, potential impact, and sustainability. The evaluation should provide credible, useful, evidence- based information which enables timely incorporation of its findings, recommendations and lessons into decision making processes of UNDP and key stakeholders. The evaluation will cover the time span from 10 November 2017 (the beginning of the Rural Housing Project) to date.

To meet the above objectives of the evaluation, the Chief, Strategic Partnerships and State Outreach/ Junior Professional Officer will work together with a national consultant throughout the assignment.

E. Scope of Work and Timeline:

The scope of work for the National Consultant for this evaluation will include but  not be limited to:

• Develop and finalize the inception report that will include elaboration of how each evaluation question will be answered along with proposed methods, proposed sources of data, and data collection and analysis procedures.

• Designing of tools and data collection.

• Undertake data collection, analysis, and interpretation.

• Write the draft evaluation report.

• Present the initial findings 

• Finalize the evaluation report after receiving feed back from UNDP and government

Devoirs et responsabilités

F. Evaluation Questions:

The evaluation questions define the information that must be generated as a result of the evaluation process. The answers will provide key basis to the intended users of the evaluation in making informed decisions, taking actions, or adding knowledge. Some of the tentative questions can be as follows:

Relevance: The extent to which the objective, purpose and outcomes of the intervention are consistent with the needs and interest of the people and the needs of the country.

• To what extent was the rural housing project supportive in proper implementation to ensure houses in the rural odisha as per the government rural housing scheme 

• To what extent was the design and strategy of the rural housing relevant to the national and UN priorities for poverty alleviation in India?

• To what extent was the design and strategy of the rural housing aligned with CPD (2018-2022) and UNDAF (2018-2022)?

• To what extent was the theory of change applied in the rural housing project relevant to serve the needs of the country?

• To what extent did the rural housing project align itself with the National Development Strategies and/or the UNDAF India?

Effectiveness: Extent to which the outcomes of the development intervention have been achieved 

• To what extent has the project achieved the objectives and targets of the results framework in the Project Document?

• Compared to 2017, to what extent the key stakeholders can deliver the same in the field? To what extent can any changes linked to rural housing strategic interventions be attributed to the project?

• What factors contributed to the achievement or non-achievement of the rural housing outcomes and outputs?

• To what extent and in what ways has ownership - or the lack of it - by the implementing partner impacted on the effectiveness of the rural housing project?

Efficiency: Extent to which resources/inputs (funds, time, human resources, etc.) have been turned into results.

• To what extent were the rural housing outputs delivered in time to ensure high quality?

• To what extent has rural housing ensured value for money?

• To what extent was resource mobilization efforts successful? Was funding sufficient for      achievement of results? (Funding analysis)

• To what extent and in what ways has ownership - or the lack of it - by the implementing partner impacted on the efficiency of the rural housing project?

• To what extent was there any identified synergy between UNDP initiatives/projects that contributed to reducing costs while supporting results?

• How well did project management work for achievement of results?

• To what extent did project M&E systems provide management with a stream of data that allowed it to learn and adjust implementation accordingly?

Sustainability: Probability of the benefits of the intervention continuing in the long term

• To what extent will the rural housing project achievements can be sustained? What are the indicators of sustainability for these achievements, e.g., through requisite capacities (systems, structures, staff, etc.)? What are the challenges and opportunities?

• To what extent are policy and regulatory frameworks in place that will support the continuation of rural housing project?

• To what extent are the institutional mechanisms in place to sustain impacts of rural housing project?

Evaluation of Cross-Cutting Issues:

• Leave no one behind and gender aspects will be considered well in evaluation questions as well the evaluation process. Gender analysis, including gender disaggregated data need to be incorporated in the evaluation.

Leave no one behind:

• To what extent have the research and monitoring of rural housing been inclusive in terms of capturing the situation of the most vulnerable and marginalized.

Gender Equality:

• To what extent has rural housing staff and other key stakeholders’ capacity been strengthened for promoting and protecting women’s rights.Is there gender marker data assigned to this project representative of reality?

• To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

Way forward

• Have any good practices, success stories, lessons learned, or transferable examples been i identified, please describe, and document them.

• Based on the achievements to the date, provide forward looking programmatic recommendations for UNDP engagement in housing sector in the next country programme document.

G. Methodology

It is strongly suggested that the evaluation should use a mixed method approach – collecting and analyzing both qualitative and quantitative data using multiple sources to draw valid and evidence-based findings and conclusions and practical recommendations. The methodology proposed should be gender responsive that is able to measure the degree of gender inclusion in within the scope of this project.  

The evaluation consultant (s) is    expected not only to conduct specific surveys to collect quantitative/qualitative data but also is highly encouraged to review all relevant reports providing quantitative data collected by rural housing. However, the evaluation consultant (s) is/are expected to propose and determine a sound evaluation design and methodology (including detailed methodology to answer each evaluation question) and submit it to UNDP in the inception report following a review of all key relevant documents and meeting with UNDP and GoO. 

Final decisions about the specific design and methods for the evaluation will be made through consultation among the GoO, UNDP, the consultants, and key stakeholders about what is appropriate and feasible to meet the evaluation purpose and objectives as well as answer the evaluation questions, given limitations of budget, time and data. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with stakeholders. All stakeholder meetings will be organized virtually for primary data collection given the scarcity of time.

Methods to be used by the evaluation team to collect and analyze the required data shall include but not limited to:

a) Desk Review: This should include a review of inter alia

• Project document

• Result Framework/M&E Framework

• Project Quality Assurance Report

• Annual Work Plans

• Annual Reports

• Highlights of Project Board meetings

• Studies relating to the country context and situation

b) Interviews:

Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed

i) Key informant interviews with relevant stakeholders from government agencies and CSOs 

ii) Semi-structured interviews with key stakeholders including UNDP, Government partners, UN & Government local staff members. 

• Development of evaluation questions around relevance, effectiveness, efficiency, and sustainability and designed for different stakeholders to be interviewed

• Analysis of RH project funding, budgets and expenditure generated from Atlas.

• Analysis and interpretation of qualitative and quantitative data available from various credible sources like AWAS SOFT

• Data review and analysis of monitoring and other data sources and methods: ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources

Data and evidence will be triangulated with multiple sources to address evaluation questions. The final methodological approach including interview schedule and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between UNDP, GoO stakeholders and the consultant(s).

H. Gender and Human Rights-based Approach

As part of the requirement, evaluation must include an assessment of the extent to which the design, implementation, and results of the project have incorporated gender equality perspective and rights-based approach. The evaluators are requested to review UNEG’s Guidance in Integrating Human Rights and Gender Equality in Evaluation during the inception phase .

In addition, the methodology used in the final evaluation, including data collection and analysis methods should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. Detailed analysis on disaggregated data will be undertaken as part of final evaluation from which findings are consolidated to make recommendations and identify lessons learned for enhanced gender- responsive and rights-based approach of the project.

These evaluation approach and methodology should consider different types of groups in the rural housing project intervention – women, youth, minorities, and vulnerable groups.

I. Expected Deliverables

The national consultant will be responsible for ensuring the following outputs/deliverables to UNDP India as per the agreed work plan. The consultant ensures that gender aspects mentioned in this ToR are included in all deliverables:

i) Inception Report:

The consultant(s) will commence the evaluation process with a desk review and preliminary analysis of the available information provided by UNDP. Based on the ToR, initial meetings with the UNDP and the desk review, the consultants should develop an inception report which will be around 5 pages in length and will elaborate evaluation methodologies, including how each evaluation question will be answered along with proposed methods, proposed sources of data, and data collection and analysis procedures. The inception report will include the evaluation matrix. UNDP and GoO will review the inception report and provide useful comments for improvement. This report will serve as an initial point of agreement and understanding between the consultant and UNDP.

ii) Draft Evaluation Report:

The evaluation report will contain the same sections as the final report and shall follow the structure outlined in Annex 3/ Evaluation Report Template and Quality Standards (Page 49-53) of Section 4/ Evaluation Implementation of UNDP Evaluation Guideline (2019) . The draft report will be reviewed by the GoO and UNDP. The draft report will ensure that each evaluation question is answered with in-depth analysis of information and back up the arguments with credible quantitative and/or qualitative 

http://www.uneval.org/papersandpubs/documentdetail.jsp?doc_id=980

  Evaluation Report Template and Quality Standards of UNDP Evaluation Guideline (2019), Section 4 : Evaluation Implementation, available at http://web.undp.org/evaluation/guideline/evidence.

iii) Presentation/Debriefing/Audit Trial: A meeting will be organized with key stakeholders including UNDP and GoO to present findings, conclusions and recommendations.

iv) Final Evaluation Report: The final report will incorporate comments and feedbacks from the stakeholders including the feedback provided during the Presentation/Debriefing meeting. Other relevant documents (i.e., data collection tools, questionnaires, datasets, if any) need to be submitted as well.

v) Evaluation Brief: A concise summary of the evaluation report will include findings, conclusions and recommendations using plain language targeting wider audience. This concise summary will be not more than 4 pages.

J. Price Proposal and Schedule of Payments

Consultants must send a financial proposal based on Lump Sum Amount. The total amount quoted shall be all-inclusive and include all costs components required to perform the deliverables identified in the TOR, including professional fee, travel costs, living allowance (if any work is to be done outside the IC´s duty station) and any other applicable cost to be incurred by the IC in completing the assignment. The contract price will be fixed output-based price regardless of extension of the herein specified duration. Payments will be done upon completion of the deliverables/outputs and as per below percentages:

The expected outputs, deliverables and payment schedule is as follows:

K. Travel:

No travel will be required for this assignment. It will be homebased; all meetings and interviews will be virtual online. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and Individual Consultant, prior to travel and will be reimbursed.

L. Implementation Arrangement, Supervision and Performance Evaluation:

This evaluation is commissioned by UNDP India. The Deputy Resident Representative, UNDP India will be responsible for managing the evaluation throughout the entire process and Rural Housing section will provide necessary support in day-to-day operation of evaluation. The consultant will report to and work under supervision of the Chief, Strategic Partnership & SDG Integration.

M. Evaluation Team Composition and Required Competences

The evaluation team will be comprised of two national consultants. One Junior Consultant to do background writing, data crunching and report writing and one Senior Consultant with substantial expertise in rural housing to design questions, do interviews and prepare policy oriented recommendations.   

Compétences

Corporate Competencies:

• Demonstrates integrity by modeling the UN’s values and ethical standards (human rights,

tolerance, integrity, respect, and impartiality);

• Promotes the vision, mission, and strategic goals of UNDP;

• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Functional Competencies:

• Consistently approaches work with energy and a positive, constructive attitude ;

• Strong interpersonal and written and oral communication skills;

• Strong analytical skills and strong ability to communicate and summarize this analysis in writing

• Has ability to work both independently and in a team, and ability to deliver high quality work on tight timelines.

Skills:

• Strong planning skills

• Excellent written and presentation skills (English)

• Strong communication skills

• Strong supervision and guiding skills

• Ability to deliver under pressure/meet deadlines

• Ability to network with partners on various levels

• The necessary computer skills with competence in MS office package

Qualifications et expériences requises

Qualifications:

• A masters’ degree or equivalent, social sciences, rural development, or other relevant fields.

• At least 10 years of working experience in the field along with 5 years of rural housing

• Experience in conducting at least 10 evaluations or assessment of large-scale policies and programs on large scale projects in development sector.

• Possess strong analytical and writing skills, with the ability to conceptualize, articulate, write, and debate about development including housing issues.

• Experience in implementing a range of qualitative and quantitative data collection tools and methods in project evaluation.

• Knowledge of current issues and innovation in results-oriented monitoring, including trends, principles and methodology.

• At least 5 years of experience in supervision

• Good knowledge of UN and/or UNDP’s mandate and socio-political context in the region.

• Advanced level of proficiency in both written and spoken English.

N. Evaluation Ethics

The evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ . The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

O. Conflict of Interest

The consultants cannot have participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) and should not have a conflict of interest with project’s related activities.  

P. Evaluation of the proposals:

Evaluation Method and Criteria

Individual consultant will be evaluated based on the following methodology:

Cumulative analysis

The award of the contract shall be made to the individual consultant whose offer has been evaluated and determined as a) responsive/compliant/acceptable; and b) having received the highest score out of set of weighted technical criteria (70%). and financial criteria (30%). Financial score shall be computed as a ratio of the proposal being evaluated and the lowest priced proposal received by UNDP for the assignment.

Technical Criteria for Evaluation for International Consultant (Maximum 70 points)

Only candidates obtaining a minimum of 49 points (70% of the total technical points) would be considered for the Financial Evaluation.

Financial Evaluation (Total 30 marks)

Financial Proposal:

Technically qualified consultants will be requested to submit their lump sum rate i.e. consultants who score more than 70% i.e. 49 marks with respect to the above-mentioned evaluation criteria. Consultant should not specify their consultancy fee on their CV or with the submission. The CV will not be evaluated further in case the consultant submits the same.