Consultant – Evaluation Specialist: Evaluability Assessment of Child and Youth-Centered Recovery Programme in Ukraine, Kyiv, 3 months (Mixed type: office based and work from home)

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential. 

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone. 

And we never give up. 

For every child, hope

UNICEF Ukraine

Full text of TOR is attached: 

Terms of Reference UKR KfW Evaluability Assessment Clean (Assistant Eval Specialist) full ver for advertisement.pdf

How can you make a difference? 

2. Purpose of assignment:

There are two main purposes of this EA:

1. Maximizing the likelihood of achieving objectives of “Child and Youth-Centered Recovery in Ukraine” Programme, by identifying areas for reinforcing or adjusting its M&E system, the programme approaches and intervention logic, identifying data sources and gaps and by fostering accountability and results ownership among the Programme stakeholders (proactive)
2. Preparing a future summative evaluation: by assessing the readiness of the Programme to be evaluated. The supporting purpose is to improve the Theory of Change that the Programme is based on and identify the factors needs to be addressed that will allow for a meaningful evaluation of the contribution of UNICEF’s work to wider recovery effort and long-term sustainable development in Ukraine (forward-looking). The EA should result in recommendations to improve the proposed approaches and design of the summative evaluation, including the potential of impact assessments.

It is expected that EA findings and lessons will inform wider stakeholders planning and/or implementing similar interventions both within and outside Ukraine for their improved programming and coordination.

3. Scope of Work:

1. Scope of Work:

The scope of this EA will encompass the following key areas:

Thematic focus

3.1 Programme Design

• Theory of Change (ToC): Assess the clarity and coherence of the programme’s ToC, including the logical flow from inputs and activities to outputs, outcomes, and impacts. Assess whether the ToC adequately reflects the complexities of the environment in Ukraine and whether it incorporates assumptions, risks, and external factors that could influence programme outcomes.
• Results Framework: Review the programme’s results framework, including the alignment of outputs, outcomes, and impacts with the overall objectives of the Child and Youth-centred Recovery Programme in Ukraine. Assess whether the results chain is coherent and whether the linkages between different levels of the framework are logical and plausible.
• Alignment with Strategic Priorities and Equity Focus: (a) Analyse the alignment of the programme’s objectives and results with Ukraine’s national recovery priorities (including on gender equality), strategic goals of UNICEF Country Programme 2025-2029 and UN Sustainable Development Cooperation Framework (UNSDCF) 2025-2029, and relevant international frameworks such as the Sustainable Development Goals (SDGs). (b) Examine whether the programme and its M&E system are adequately designed and targeting the vulnerable and marginalized groups.

3.2 Monitoring and Evaluation Systems

• M&E Framework: Assess the robustness of the M&E framework, including the quality of indicators, the presence of baselines and targets, relevance of means of feasibility and the mechanisms for data collection, analysis, and reporting. Analyse whether the framework is capable of tracking progress across all levels of the results chain and whether it allows for timely and accurate measurement of outcomes.
• Data Collection Tools and Systems: Review the tools and systems in place for collecting, storing, and analysing data. Assess the adequacy of these systems in terms of their ability to generate reliable, timely, and disaggregated data, particularly in the context of a complex and dynamic ongoing war environment. Identify the gaps is these systems and propose areas to be strengthened during the early phase of the Programme.
• Capacity for M&E: Assess the capacity of programme staff and implementing partners to effectively carry out M&E activities. This includes assessing the availability of human and financial resources, the level of M&E expertise, and the adequacy of training and support provided to staff and partners.
• Models for Replication and/or Scale-up: assess the appropriateness and feasibility of the identified recovery models and interventions for replication and/or scale-up. Review the appropriateness and relevance of each model identified and of its indicators for capturing success, the alignment and coherence with the intended Programme outputs and outcomes as well as examine the adequacy and feasibility of its means of verification, data collection process, tools and resources. Identify the gaps and provide suggestion on measures to strengthen its relevance and ability to demonstrate replicability and/or scalability of pilot models.

3.3 Data Availability and Quality

• Baseline Data: Assess the availability and quality of baseline data for all key indicators. Determine whether the baselines are comprehensive, up-to-date, and reflective of the current context in Ukraine.
• Data Sources: Review the sources of data used to monitor programme performance, including administrative data, surveys, and other data collection methods. Examine the reliability, validity, and timeliness of these sources, and identify any gaps or weaknesses.
• Data Quality Assurance: Assess the processes and systems in place to ensure the quality of data collected, including data verification and validation procedures, data management practices, and the use of technology in data collection and analysis.

3.4 Evaluation Readiness

• Stakeholder Engagement: Assess the extent to which UNICEF’s key internal stakeholders and UNICEF’s partners have a shared understanding of what the Programme attempts to accomplish, and how it is achieved. Determine whether there are mechanisms in place for stakeholder feedback and participation in the monitoring and evaluation process. Identify key stakeholders that needs to be involved in Programme implementation and M&E process.
• Resource Allocation: Review the allocation of resources for evaluation activities, including budgetary provisions for M&E, the availability of M&E expertise, and the adequacy of logistical support.
• Contextual Considerations: Consider the broader context in which the evaluation will take place, including the political, social, and security environment in Ukraine. Assess whether these factors could influence the feasibility, credibility, and utility of robust monitoring and the summative evaluation. Identify the potential risks and mitigating measures.
• Potential Risks: identify potential risks that could hinder the likelihood of the Programme interventions to make expected contribution to the intended results achievement at different levels.
• Feasibility of assessing impact: examine the extent to which assessment of programme’s contribution to impact-level results would be feasible, given the current programme design (including the ToC and results framework), M&E framework and data availability. EA is also expected to identify the gap to address for the forthcoming evaluation to be ready to assess the contribution to mid-term and long-term impact.

Time-period

The EA primary examines the design, framework and intervention logic of the current Programme covering the period from January 2024 until the end of 2026. The EA should also take into account the period of the preceding project since November 2022.

Geographic scope

The EA will principally focus on 18 priority hromadas covered in the current Programme started in January 2024, and, where relevant, 36 priority hromadas covered in the preceding project since November 2022 (including 8 hromadas covered in the current Programme) will be covered.

It is not in the scope of this EA to provide cost-effectiveness or cost-benefit analysis of the planned interventions nor to collect the baseline data for key indicators of the Programme as it is covered in a separate exercise.

4. Methodology

In the proposal, the applicant(s) will be expected to suggest a comprehensive and realistic evaluability assessment approach which should be further developed by the selected team and finalized in consultation with UNICEF evaluation specialist during the inception stage. The applicant(s) are encouraged to elaborate on these suggested approaches and methods. Yet they may propose alternative approaches and methods if deemed more appropriate and with clear justification.

5. Timeframe and key deliverables

1. Timeframe and key deliverables

The list below outlines the breakdown of the expected deliverables of the EA at different phases:

Phase 1. Inception Phase: September – early October 2024

• A kick-off meeting to explain the basic concepts of the evaluability assessment, the evolution of UNICEF Recovery Programme and their linkages to new UNICEF CPD 2025-2029;
• A draft and final Inception report (approx. 15 pages excluding annexes), including purpose, objectives, scope, suggested methodology (including approaches, data collection tools and analysis methods, ethical considerations and mitigating measures); finalized list of questions; work plan; timeline/schedule to conduct interviews and data collection; timeline for validation workshops, and an outline for the final EA report.

Phase 2. Data collection and analysis phase – October 2024

• A power point presentation on preliminary EA findings for the EA Reference Group (ERG) and selected members of the UNICEF CO and partners

Phase 3. Validation and finalization – November 2024

• A first draft evaluability assessment report (max 40 pages including the executive summary and excluding annexes) to be validated by the ERG and selected members of the UNICEF CO.
• A workshop with key internal and external stakeholders at which the draft EA report will be presented, and feedback sought on factual errors within the report, errors on interpretation, and appropriateness of conclusions. An adapted power point presentation (in English and in Ukrainian) will need to be developed for this workshop.
• A final EA report based on comments and feedback received on the draft report during the validation phase. The proposals for appropriate evaluation approach, methodology and key questions for the planned summative evaluation should be an annex to the final report, which would help UNICEF to eventually prepare the evaluation ToR. The final report will be translated into Ukrainian for sharing with the Ukrainian counterparts. The EA report structure is expected to adhere UNICEF-adapted UNEG Evaluation Report Standards as much as possible. Exact structure of the final report is to be agreed during the Inception Phase.
• An EA Brief designed for dissemination of findings to key stakeholders. The brief should adhere UNICEF Brand book.

6. Management and Governance Arrangements

Supervisor/Approving authority: An EA Manager in the Evaluation Section of UNICEF Ukraine CO will manage and supervise the exercise. Together with EA manager, the UNICEF Ukraine CO Recovery Programme manager will also be involved in day-today oversight and regular communication with the consultant(s).

Reference Group (RG): A Reference Group (RG) will be created to support the consultants and the supervisor. The roles of the RG will be: Offer insights on issues under discussion, especially in the inception phase where the methods, design and data to be sought are to be determined; Review all deliverables produced, including the inception, emerging evidence summary presentations, and final report; Participate in workshop-style meetings for presentation and discussion of emerging findings, for validation and further comments.

7. Evaluability Assessment Team: Responsibilities

This EA will be carried out by a team of two consultants: a Senior Evaluation Specialist as Team Leader and another Evaluation Specialist as supporting member. It is expected that two members are working closely together throughout entire process of the EA. Expected responsibilities of members are as follows:

Evaluation Specialist expected responsibilities: the Team Member / Evaluation Specialist will support the Team Leader / Senior Evaluation Specialist, providing substantive feedback, based on experience and skills that complement the team leader’s, particularly on thematic areas.

Selected candidate for this Team Member/Evaluation Specialist role is expected to work closely with Senior Evaluation Specialist/ Team Leader.

Expected responsibilities of the Team Leader / Senior Evaluation Specialist are: to direct all parts of the EA, being directly accountable to UNICEF; to coordinate and supervise the work of the EA team in their contributing roles; to ensure the quality of the process, outputs, methodology and timely delivery of all products; to take direct responsibility for all deliverables being of satisfactory quality; to ensure that the deliverables emerge in a timely fashion, following a high quality in-depth analytic process, as well as ongoing consultation with the UNICEF EA Manager and the Reference Group.

Work assignment overview:

Deliverable

Deadline

# of w/days

Final Inception Report – Annexes (including detailed methodology notes, data collection tools, and preliminary document review)

04/10/ 2024

10

Power point presentation on preliminary EA findings

31/10/2024

15

Executive Summary and Annexes of the Draft Report

15/11/2024

10

Final Report Annexes and PowerPoint Presentation

15/12/2024

11

Complete offline Secure Approaches in the Field Environment (SSAFE) and Individual First Aid Kit (IFAK) training upon request

4

Sub-total:

50

Travel plan: Round trip to Ukraine.

Within Ukraine: 19 days.

• The consultant is expected to have filed visits for working meetings with hromada representatives within Ukraine (TBD Kharkiv, Dnipro, Odesa, Mykolaiv, Lviv, Kyiv). The total number of trip days per contract – 15 days – dates and hromadas should be agreed with UNICEF upon the trip.
• Security training in Lviv, Ukraine – 4 days.

Please provide an all-inclusive financial proposal including travels to carry out the deliverables listed above.

Should applicant wish to team up with a particular individual as Team Leader, the concerned individual should strongly be encouraged to apply for Senior Evaluation Specialist/Team Leader position, which is being advertised at the same time as this consultancy.

The applicant(s) are strongly encouraged to submit a brief proposal (3-5 pages) suggesting a comprehensive and realistic evaluability assessment approach.

To qualify as an advocate for every child you will have… 

Education: Fi First advanced degree [Master’s Level] in a relevant field across the social sciences, with an advantage for degrees or major emphases in evaluation or related evidence fields.

Professional experience:

• A recent work history in Ukraine, or in the region with work conducted in UN or international development environments.
• More than 5 years of professional experience in evaluations, research, analysis of organizational/institutional effectiveness issues is a strong advantage. Prior experience on both humanitarian and development programme evaluations is likewise a major advantage.
• Experience participating as a mid-level or senior team member in an institutional effectiveness evaluation, preferably within the UN system.
• Experience on conducting evaluability and/or formative exercises of complex multi-sectoral programmes and of recovery/ reconstruction interventions is a major advantage
• Clear understanding of UNICEF’s institutional structure as a decentralized organization, and the ability to analyze how global initiatives can be implemented through this decentralized structure.
• Familiarity with humanitarian-development nexus evaluations and/or recovery programme evaluations are major advantage.
• Experience in conducting analysis on feasibility of pilot replication and/or scaling up are major advancatge.

Languages: 

• English: Proficient/Native [C2];
• Ukrainian: is an advantage.

Skills and competencies: 

• Ability to support with documentation review and data gathering, including the management of interviews/consultations with key informants.
• Ability to coordinate with diverse stakeholders of both international and Ukrainian partners.
• Integrity and demonstrated ability to work with the evaluation team leader
• Excellent written and oral communication skills in English. Knowledge of Ukrainian is strongly preferrable.

Abilities to:

• analyse M&E systems in terms of indicators and systems SMARTness.
• lead participatory methods with stakeholders.
• construct and review ToCs to a suitable level of complexity.
• identify risks affecting programme performances and mitigating measures.

For every Child, you demonstrate… 

UNICEF’s values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS). 

To view our competency framework, please visit here. 

UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to include everyone, irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.

UNICEF offers reasonable accommodation for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment. 

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check. 

Remarks: 

Only shortlisted candidates will be contacted and advance to the next stage of the selection process. 

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws. 

The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts. 

Read Full Description