Background
The evaluation will analyze and rate the relevance, effectiveness, efficiency, impact and sustainability of the project results and overall performance using the following scale:
Excellent -1 – Good -2 – Satisfactory-3 – Unsatisfactory- 4 – Not applicable-5 – Insufficient information-6
The evaluation will identify overall project management main findings and key lessons including examples of best practices for future projects in the country, region and UN agencies and UNTFHS. The results of the evaluation will be reviewed by the UNTFHS.
The findings, conclusions and recommendations generated by the TE will be presented to the United Nations Country Team (UNCT) and the GoM at the end of the JP for validating or filling the gaps in the initial assessment of relevance, effectiveness, efficiency, impact and sustainability incontribution of JP to the Mongolian MDGs and major development frameworks, such as the UNDAF and Regional Development Strategy.
- To assess the extent the JP has contributed to solving the needs and problems identified in the feasibility study and/or baseline survey;
- To ascertain the degree of implementation, efficiency and quality delivered on outputs and outcomes against what was originally planned or subsequently officially revised;
- To measure to what extent the JP has attained development results for the targeted population, beneficiaries, participants including individuals, communities and institutions using the logical framework;
- To assess the joint project contribution to the objectives set in the Comprehensive National Development Strategy based on Education, Health, Livelihood, Water Supply and Sanitation MDG at national and local levels, and;
- To identify lessons learned and good practices as a result of the JP implementation within the various programme components with the aim to support the sustainability of the JP or some of its components.
Scope, Levels of Analysis and Evaluation Criteria
The TE shall focus on assessing results and impacts generated by the JP, based on the scope and criteria included in the terms of reference. The TE will be guided by a set of questions grouped according to the evaluation criteria. These criteria are grouped according to the three levels of the JP.
Design Level:
Relevance: The extent to which, the objectives of a development intervention are consistent with the needs and interest of the population or communities and the need of the country.
- How much and in what ways did the JP contributed to solving the socio-economic needs and problems identified in the design phase?
- To what extent the JP was designed, implemented, monitored and evaluated jointly?
- To what extent the JP was the best option to respond to development challenges stated in the Project Document?
- Did the JP stakeholders and target groups find the project activities useful?
- To what extent the implementing partners participating in the JP had an added value to solve the development challenges in the Project Document?
- Is there synergy or complementary between the JP interventions and that of other development partners?
- To what extent did the JP have a useful and reliable M&E strategy that contributed to measuring development results?
- To what extent have the participating UN agencies contributed to raising the quality of the design of the JP?
- To what extent participating UN agencies have acted within the framework of joint programming?
- If the JP was revised, did it reflect the changes that were needed?
- How people-centered and gender-responsive approach is ensured in the design of JP?"
- Compare the socio-economic situation and enabling environment in the design phase with the implementation phase. To what extent did this situation impact the results? What lessons did we learn from this?
Process Level:
Efficiency: Extent to which resources/inputs (funds, time, human resources, etc) have been turned into results
- To what extent was the JP management model (i.e., instruments; economic, human and technical resources; organizational structure; information flows; decision making in management) efficient in comparison to the outputs delivered?
- To what extent was the implementation of the JP intervention (group of agencies) more efficient (or less efficient) in comparison to what could have been achieved through a single agency’s intervention?
- To what extent the governance of the JP at the national and local levels contributed to efficiency and effectiveness of the JP? To what extent these governance structures were useful for development purposes, ownership, for working together as one? Did they enable management and delivery of outputs and results?
- What type of work methodologies, financial instruments, and business practices have the implementing partners used to increase efficiency in delivering as one?
- What type of obstacles (administrative, financial, technical and managerial) did the JPWS face and to what extent have this affected its efficiency?
Ownership in the process: Effective exercise of leadership by the country’s national/local partners in development interventions
- To what extent did the targeted population, citizens, participants, local and national authorities make the JP their own, taking an active role in it? What models of participation have driven the process?
- To what extent and in what ways has ownership or the lack of it, impacted the efficiency and effectiveness of the JP?
Result Level:
Effectiveness: The extent which the objectives of the development intervention have been achieved
- To what extent did the JP contribute to the attainment of outputs and outcomes initially expected in the Project Document?
- To what extent were the JP outputs and outcomes synergetic and coherent to produce development results? What kinds of results were reached?
- To what extent did the JP have an impact on the targeted population?
- Do the products created live up to the necessary quality?
- What was intervention coverage? Were the planned geographic area and target group successfully reached?
- What were constraining and facilitating factors and the influence of the context on the achievement of results?
- In what way has the JP come up with innovative measures for problem solving?
- What good practices or successful experiences of transferable examples have been identified? Please describe and document them;
- To what extent has the JP contributed to the achievement of national ownership processes and outcomes (the design and implementation of National Development Plans, Policies, UNDAF, MGDs, etc)?
- To what extent did the JP help to increase stakeholder dialogue and/or engagement on development issues and policies?
- To what extent people-centered and gender-responsive approach was achieved and what difference did it make? Any good practice?
Sustainability: Probability of the benefits of the intervention continuing in the long term
- Was the JP supported by national and/or local institutions?
- Are these institutions showing technical capacity and leadership commitment to keep working with the JP and to repeat it?
- Have operating capacities been created and/or reinforced among national partners?
- Did the partners have sufficient financial capacity to keep up the benefits produced by the JP?
- To what extent the JP decision making bodies and implementing partners have undertaken the necessary decision and course of actions to ensure the sustainability of the effects of the JP?
- Are stakeholders ready to continue supporting or carrying out specific JP activities?
- Did stakeholders replicate the activities in other regions or sectors of the country?
- Did stakeholders adapt JP results in other contexts?
- To what extent will the JP be replicated or scaled up at national or local levels?
- Did the JPWS design take into account strategies to ensure sustainability? Were strategies used in from the beginning of JP implementation? Was there an adequate strategy for capacity building?
Duties and Responsibilities
Methodology
For the TE, methodologies and techniques as determined by the specific needs for information, the questions set out in the TOR will be used as of the availability of resources and the priorities of stakeholders. In all cases, consultants are expected to analyze all relevant information sources, such as annual progress reports, project documents, project files, mission reports, strategic country development documents and any other documents that may provide evidence on which to form judgements.
Consultants are also expected to use interviews, surveys or any other relevant quantitative and qualitative tools as means to collect data for the TE. The evaluation team will make sure that the voices, opinions, and information of targeted citizens and participants of the JP are taken into account.
The methodology and techniques to be used in the TE should be agreed upon with UNDP and other stakeholders and clearly outlined and described in detail in the terminal evaluation report, and contain at the minimum, information on the instruments used for data collection and analysis, whether these be documents, interviews, field visits, questionnaires or participatory techniques.
The consultant team is expected to deliver the following deliverables to the chair of the JP Committee for further review and comments by the government and UN partner agencies (UNDP/NPC):
Inception Report: One week prior to the evaluation mission in order to clarify and get endorsement of the timing and evaluation methodologies.
Draft Final Report: By 01 September (to be submitted within 10 days after the completion of the field visits). The draft final report will contain the same sections as the final report and will be no more than 50 pages in length. It will be written both in English and Mongolianand will also contain an executive summary of no more than 5 pages that includes a brief description of the JP, its context and current situation, the purpose of the evaluation, its methodology and its major findings, conclusions and recommendations.
Terminal Evaluation Final Report: By 27 September(to be submitted within 7 days after receipt of the draft final report with comments). The final report will be no more than 50 pages in length. It will be written both in English and Mongolianand contain an executive summary of no more than 5 pages that includes a brief description of the JP, its context and current situation, the purpose of the evaluation, its methodology and its major findings, conclusions and recommendations.
The international consultant is expected to work intermittently between August and October2013, which involves three weeks mission (Ulaanbaatar and field visits) and three weeks desk work. It is expected that the evaluation team will conduct field visits to selected 3 JP target soums of 3 aimags. The Programme Officers and the PIU staff will accompany the field visit to facilitate the consultants.
Duration of the assignment is 6 weeks. Proposed time allocations are below.
Preparation and inception report including mission plan, evaluation methodologies, outline of the report:
- 26-30 August 2013 (5 working days).
- Within 02-18 September 2013 (15 working days).
- Within 16-23 September 2013 (5 working days, 2 days overlapping with the mission).
- 30 September - 04 October 2013 (5 working days).
Payment Modality and Schedule
The UNDP standard method of payment is output-based lump-sum scheme and the payment will be made in two installments upon satisfactory completion of the following deliverables:
Details of Payment and Schedule
- 50% of the total contracted amount upon the approval/clearance of the Draft Final Report, by last week of September 2013.
The 2nd Installment:
- 50% of the total contracted amount upon approval/clearance of the completion of all services, including clearance of the Final Report, by second week of October 2013.
Application Procedure:
All applications including P11 form, CV, technical and financial proposals should be directly posted on www.jobs.undp.org. This email address is being protected from spam bots, you need Javascript enabled to view it till 6:00p.m. of15 July 2013. Incomplete applications will be excluded from further consideration. For further information please contact via bids.mn@undp.org.
Please note that only one file can be uploaded in the system, so that all required supporting documents should be in one file.
Recommended Presentation of Proposal:
Introduction about the consultant/achievement based detailed CV; Proposed methodology and workplan (max 1 page); financial proposal, including proposed fee and all other international travel related costs (such as flight ticket, per diem, etc). Please kindly note that the standard for all travel authorized by UNDP for individual subscribers is economy class.
Criteria for Evaluation of Proposal:
The selection will be made based on the educational background and experience on similar assignments. The price proposal will weigh as 30% of the total scoring.
Competencies
- Demonstrates a strong commitment to UNDP mission, vision and values;
- Displays cultural, gender, religion, race nationality and age sensitivity and adaptability;
- Treats all people fairly without favoritism and integrity.
- Have good communication skills and builds strong relationship with clients, focuses on impact and result for the client and responds positively to feedback;
- Familiarity with project management operations and procedures of various donor agencies and/or multi-lateral institutions and United Nations, M&E procedures and UNTFHS guidelines preferred;
- Ability to accommodate additional demands on short notice.
Required Skills and Experience
Education:
- Post graduate degree (preferably Ph.D. or D.Phil.) in economics, natural resource management, water supply and sanitation, health, education, livelihood, human security or related fields.
Experience:
- At least 10 years of work experience in respective expertise areas;
- Experience with result-based management evaluation methodologies;
- in applying participatory monitoring approaches; and SMART indicators and reconstructing or validating baseline scenarios;
- Experience in evaluation of similar projects/programmes and cold climate countries will be an asset;
- Evaluators must be independent, not have been engaged in the activities to be evaluated, or responsible in decision making roles for the design, implementation or supervision of the JP.
- The evaluation team should comprise of one International Expert (Team Leader) and one National Expert with strong evaluation experience, with requisite expertise in the subject matter of the project, and with expertise in economic and social development issues.
Skills/knowledge
- Demonstrated experience and ability to reliably contribute to output- and outcome-based evaluations;
- Strong analytical and writing skills with proven record in M&E, advocacy, policy recommendations and problem identification and solving.
Language:
- Excellent ability to work in English, effective oral and written communication skills.