International Consultancy : Artificial Intelligence consultant for Country Programme Evaluation

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.

And we never give up.

For every child, Future

A. Background:

UNICEF supports vulnerable children and women in over 20 countries in the Middle East and North Africa (MENA) Region. Country Offices and national governments develop multi-year Country Programmes to address child rights gaps among the most vulnerable children.

At the end of the programme cycle, Country Programme Evaluation assesses strategic aspects for accountability and learning. Managed by Regional Evaluation Advisors and conducted by external evaluation team, the evaluation inform the next programme cycle. In MENA, evaluations are planned for Libya and Palestine in 2025, and Algeria, Egypt, and Jordan in 2026.

Country Programme Evaluation covers the entire programme over multiple years. UNICEF’s internal systems collect and manage large volumes of data, but this data has been underutilized in evaluations. AI technologies, like machine learning and Natural Language Processing (NLP), can automate analyses, enhancing decision-making and understanding of programme results.

AI use in UNICEF evaluations is emerging, aiming to improve accuracy, efficiency, and depth. NLP has been tested for identifying themes, extracting insights, and detecting trends in evaluations. It also transcribes and analyzes interviews and meetings, and conducts sentiment analysis on social media data. The MENA Regional Office is piloting for an evaluation synthesis. The office is now planning to expand AI use in Country Programme Evaluations.

B. Objectives of Assignment: 

The purpose of this assignment is to facilitate and enhance the use of data and information of UNICEF’s systems for the evaluation of country programmes through designing and piloting an AI approach, supporting the evaluation team in conducting analyses, and capturing this in a methodological guidance note on the use of AI in evaluations. The targeted users are the evaluation manager(s), evaluation team leaders and evaluation reference group/steering committee members of country programme evaluations. Once piloted, the methodological guidance note could also support the evaluation managers and team leaders of other evaluations in integrating the use of AI in the data collection and analysis of the evaluation.

The objectives of the assignment are to explore and pilot the use of AI in evaluations, to draft a guidance note and to provide guidance to the evaluation manager and team leader on the use of AI in Country Programme Evaluations.

This is to be done by exploring existing data and UNICEF internal systems and understanding their structures and proposing potential areas of analyses where and how AI can support in the evaluations, while building on the already existing experience in UNICEF. The incumbent will also assess existing gaps in data and capacity which would enable further analyses. During the evaluation implementation, the incumbent is to support the evaluation team in conducting analyses in coordination with the evaluation manager, depending on the needs.

C. Scope & Work Assignment

1. Identification of areas of analyses with use of AI

The incumbent will first review existing data and information systems of UNICEF, including the ones that are available on the UNICEF internal system, such as insight, eTools, and EISI, as well as in the public domain. Contrasting them with a list of predefined Country Programme Evaluation questions, the incumbent will explore and identify potential areas where use of AI can facilitate analyses in responding evaluation questions. Consequently, the incumbent will develop a table to list types of analyses and describe required data, data availability, required preparatory work, system requirements, expected outcomes of analysis, and relevant evaluation questions for each analysis, while clarifying existing gaps. It will guide actual analyses to pilot.

2. Preparation for analyses & Piloting

Following step 1, selected analyses with use of AI will be piloted to demonstrate how the outcome of analyses will contribute to the overall analyses to respond relevant evaluation questions. In preparation to such analyses, the incumbent will build architecture for data gathering and testing modelling approaches. API bridge should be developed to facilitate/automate the process. The process may include data/text cleaning and formatting. The preparation and piloting should be done in such a manner that it can be applied to multiple country programme evaluations.

3. Support the evaluation implementation

Based on the outcomes of the step 2 above, the incumbent will support the evaluation team in conducting analyses and triangulation during the implementation of a Country Programme Evaluation. The incumbent should be periodically in touch with the evaluation team member who conducts data analyses to provide necessary support by clarifying available data and expected analyses. The evaluation team may propose additional analyses based on available data and systems, for which the incumbent should support so that they can be accessible and usable by the evaluation team. The incumbent should also coordinate with the evaluation manager for the quality check, particularly for the analyses with use of AI and their interpretations. Depending on the outcome of the step 2, the support may not be needed, and this step 3 may be dropped.

4. Draft methodological guidance note addressing all 3 steps above.

The incumbent will develop a methodological guidance note which documents the processes from the previous steps. It will describe how to replicate the AI-driven analyses, including best practices for data collection, preparation, and modeling. The note will also outline the challenges encountered during the pilot phase and actual application to the country programme evaluation. The incumbent will identify existing capacity and data gaps and propose solutions to address these issues as a part of the guidance note. Apart from the guidance note, the incumbent will develop PowerPoint slides work s/he did for a Country Programme Evaluation for dissemination of the work.

The incumbent will report to Evaluation Specialist (Multi-Country) at the Regional Office.

How can you make a difference?

Phase 1: Identification of Areas of Analyses with Use of AI

Tasks/Milestones:

• Gather and review existing data sets and systems needed for the UNICEF Country Program evaluation.
• Review evaluation questions.
• Develop a table outlining potential analyses based on the available data and evaluation questions.
• Agree on which analyses to pilot.

Deliverable & Timeframe:

• A table of potential analyses linked to the evaluation questions and a proposal for the pilot model development, to be completed between December 2024 and February 2025 (10 days).

Phase 2: Pilot Testing of Analyses

Tasks/Milestones:

• Develop an API bridge, if access is granted, to pull data from relevant corporate systems.
• Build the data architecture.
• Clean the text and data for analysis.
• Conduct pilot analyses.
• Select the appropriate machine learning (ML) model.
• Train the ML model using the existing data set.
• Pilot the trained model for evaluation questions.

Deliverable & Timeframe:

• API, data architecture, cleaned text/data, and outcomes of the pilot analyses, to be completed between March and April 2025 (15 days).

Phase 3: Support the Evaluation Implementation

Tasks/Milestones:

• Clarify the data and systems available for use by the evaluation team.
• Share the deliverables developed during the preparation phase.
• Review and check the quality of analyses done by the evaluation team using AI.

Deliverable & Timeframe:

• Verified analysis outcomes, to be completed between May and October 2025 (10 days).

Phase 4: Development of Guidance Note

Tasks/Milestones:

• Document all processes related to the AI-driven analyses to provide guidance for replication.
• Develop PowerPoint slides to disseminate the work and findings.

Deliverable & Timeframe:

• A guidance note and PowerPoint presentation on the work, to be completed between September and October 2025 (7 days).

To qualify as an advocate for every child you will have…

Master’s degree in data science, computer science, artificial Intelligence, engineering, statistics, or a related field. A bachelor’s degree in the above fields combined with 2 additional years of experience may be accepted in lieu of Master’s.

• At least 5 years of professional experience, including applying AI and machine learning techniques, particularly in the context of data analysis and preferably programme evaluation.

• Proven experience in using AI technologies, including NLP and machine learning.

• Proficiency in programming languages such as Python, R, or similar, with a strong understanding of AI and machine learning frameworks.

• Strong background in data management and architecture.

• Familiarity with API development and integration for data processing and automation.

• Experience in designing and implementing AI models for qualitative and quantitative data analysis.

• Knowledge of virtual machine.

• Understanding of ethical considerations and data privacy issues related to the use of AI (familiarity with data principles, including storage and security).

• Commitment to upholding UNICEF’s values and principles in all aspects of work.

• Excellent written and verbal communication skills, with the ability to present complex technical information to non-technical stakeholders.

• Fluency in written and oral English

For every Child, you demonstrate…

UNICEF’s values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS).

To view our competency framework, please visit  here.

UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious or ethnic background, and persons with disabilities, to apply to become a part of the organization. To create a more inclusive workplace, UNICEF offers paid parental leave, breastfeeding breaks, and reasonable accommodation for persons with disabilities. UNICEF strongly encourages the use of flexible working arrangements. Click here to learn more about flexible work arrangements, well-being, and benefits.

According to the UN Convention on the Rights of Persons with Disabilities (UNCRPD), persons with disabilities include those who have long-term physical, mental, intellectual, or sensory impairments which, in interaction with various barriers, may hinder their full and effective participation in society on an equal basis with others. In its Disability Inclusion Policy and Strategy 2022-2030, UNICEF has committed to increase the number of employees with disabilities by 2030. At UNICEF, we provide reasonable accommodation for work-related support requirements of candidates and employees with disabilities. Also, UNICEF has launched a Global Accessibility Helpdesk to strengthen physical and digital accessibility. If you are an applicant with a disability who needs digital accessibility support in completing the online application, please submit your request through the accessibility email button on the UNICEF Careers webpage Accessibility jobs-near-me.org UNICEF.

UNICEF does not hire candidates who are married to children (persons under 18). UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination based on gender, nationality, age, race, sexual orientation, religious or ethnic background or disabilities. UNICEF is committed to promote the protection and safeguarding of all children. All selected candidates will, therefore, undergo rigorous reference and background checks, and will be expected to adhere to these standards and principles. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check, and selected candidates with disabilities may be requested to submit supporting documentation in relation to their disability confidentially.

Remarks: 

Only shortlisted candidates will be contacted and advance to the next stage of the selection process.

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.