BHA Program Evaluation – Consultant

SCOPE OF WORK

Organizational and program background

The International Rescue Committee (IRC) responds to the world’s worst humanitarian crises and helps people whose lives and livelihoods are shattered by conflict and disaster to survive, recover, and regain control of their future.

Since April 2023, the International Rescue Committee (IRC), in collaboration with a partner, has been delivering essential Water, Sanitation, and Hygiene (WASH), primary health, and reproductive Health (RH) services through four static facilities, and seven satellite clinics. Access to safe water and comprehensive health services remains a critical challenge in Sudan, where IRC and its partner aim to bridge this gap by providing essential safe water, sanitation, and hygiene services.

IRC and its partner have also established four maternity units offering 24/7 services, staffed with qualified professionals who provide a comprehensive range of care, including: Basic Emergency Obstetric and Neonatal Care (B-EmONC), Antenatal Care (ANC), Postnatal Care (PNC), Post-abortion Care, counseling and treatment for sexually transmitted infections (STIs), counseling and treatment for sexual assault survivors and Family Planning (FP) services.

The IRC is implementing a 21-month project entitled Life-saving WASH assistance and health care provision to cut-off populations in Sudan**.** This project runs from April 1, 2023 through December 31, 2024. Two cost modifications of the initial project were granted in April and July 2024.

The program’s goal is to improve community health through increased access to primary and reproductive health care, clean water and sanitation, and the promotion of positive health, sanitation, and hygiene practices. The goal of this project aligns with BHA’s mission by providing lifesaving humanitarian assistance to vulnerable populations in hard-to-reach locations.

The program targets IDPs and vulnerable host communities and the key objectives are as follows:

• To provide quality primary and reproductive health services to women, men, boys, and girls and increase awareness on disease prevention
• To ensure access to and demand for sustainable, adequate, and safe water sources for women, men boys and girls, and to improve hygiene and sanitation for all

Rationale and Purpose

The primary objective of this project is to deliver quality primary and reproductive health services to 297,758 individuals, while promoting disease prevention awareness for an estimated 58,644 people. Additionally, the project aims to secure access to sustainable, adequate, and safe water sources for 436,000 people and to improve hygiene and sanitation practices for 50,500 individuals.

The overall objective of this evaluation is to assess the impact of the program in reducing morbidity and mortality rates and enhancing the well-being of conflict-affected populations. The evaluation will determine the extent to which integrated programming has improved the lives of target beneficiaries and will draw lessons learned and key recommendations as for IRC’s decision-making to inform the design of future programs.

Objectives of the Evaluation

The main purpose of the evaluation is to assess the impact, relevance, sustainability, effectiveness, and efficiency of the program, applying the OECD DAC criteria. This program evaluation will include a comparative analysis of baseline and endline values for key outcome indicators, identification of program achievements and noting areas for improvement. Findings and recommendations will assist the IRC, its partner, and BHA in identifying gaps and enhancing future program design and implementation.

The consultant will collect and analyze data to determine factors contributing to success and recommend areas for improvement.

Specific Objectives

1. Relevance

a) Evaluate the program’s impact on the target population since its inception in April 2023.

b) Examine the contribution of the integrated health and WASH program approach to achieve outcomes for the target population.

2. Efficiency

a) Evaluate if the program demonstrated a good Value for Money (VfM) approach

b) Examine the efficiency of the management structures and implementation modalities in terms of delivery timeliness and cost-effectiveness of the interventions.

3. Effectiveness

a) Identify lessons learned and good practices and provide actionable recommendations to enhance program effectiveness.

4. Impact analysis

a) Assess intended and unintended outcomes of the program, offering solutions to mitigate any negative unintended effects.

5. Sustainability and Program Adaptation

a) Evaluate the impact of new population influxes on the program’s objectives and outcomes and recommend adjustments as needed.

Evaluation Criteria and Questions

The evaluation should explore the program’s impact, relevance, efficiency, effectiveness, and sustainability by critically analyzing sector-specific activities and approaches that were used to achieve the desired change in the lives of the beneficiaries. The evaluation will address the following sector-specific questions:

1. How have IRC-supported primary health care facilities improved access to comprehensive services and health outcomes for target communities?
2. Were satellite health units effective in improving access to health services, and what was their impact on community health outcomes?
3. To what extent has IRC’s medical referral system facilitated timely and impactful access to specialized care?
4. How effective were WASH interventions in improving access to safe water, sanitation, and hygiene awareness among the target population?
5. To what extent did IRC’s WASH activities address the needs of vulnerable groups and contribute to reducing WASH-related disease risks?
6. How effective were the community feedback mechanisms in ensuring inclusivity and responsiveness to community needs?

Scope of consultancy

The Consultant will design an appropriate evaluation methodology based on their understanding of the expectations of the scope of work. The consultant must physically travel to Sudan to undertake the primary data collection exercise in-person; remote work is not possible. The consultant should note that project locations are in remote areas with poor road conditions and no mobile phone network. Internet is available when at the base. The use of mobile technology (e.g., tablets for data collection) is limited by a lack of reliable electricity to charge devices for data collection. The consultant should therefore plan for flexibility in timelines and data collection methods.

The Consultant is expected to propose a methodology for the evaluation that should include but not limited to:

• Drafting data collection tools, with a plan for feedback on tools to be used in implementation contexts.
• Developing the data collection methodology, including sampling.
• Developing the quantitative and qualitative data analysis plan.
• Plan for validation of findings with key stakeholders.
• Production of the evaluation report using USAID evaluation format.

The consultant should propose a mixed methodology that combines qualitative and quantitative evaluation. Expected deliverables include a desk review, inception report with sampling framework, developing data collection tools, developing the quantitative and qualitative data analysis plan and finally producing a quality evaluation report.

All data collected must be disaggregated according to BHA requirements that include but are not limited to sex, age, disability and location. The consultant should submit a detailed inception report that includes data collection instruments, clear roles and responsibilities, timeframe, data analysis process, and report writing.

Dissemination Plan

IRC and its partner will ensure that evaluation findings are effectively disseminated to the impacted communities, relevant stakeholders, and BHA through the following dissemination plan:

1. Community members, local leaders, and stakeholders: Findings will be disseminated through a condensed, translated version of the report to ensure accessibility. Findings will be shared further during quarterly project community meetings and stakeholder meetings to facilitate understanding and actionable discussions.

2. Internal Staff: The report and PowerPoint Presentation will be disseminated to relevant staff. Findings will be used to inform future project design meetings to consolidate learning and reflect on project outcomes.

3. BHA Representatives: Evaluation report findings will be annexed to the final technical report.

Evaluation Roles and Responsibilities

IRC and its partner will:

• Provide accommodation and travel Including flights and ground transport
• Provide a vehicle to access remote sites as feasibly as possible
• Connect the consultant to a network of local field monitors for data collection and facilitate payment based on a rate and schedule pre-approved by IRC.
• Provide access to power sources for charging and recharging for electronic devices and internet connectivity for offline mobile data collection and online data submission when the team are back to the office.
• Provide a pre-departure briefing and ongoing security advice and support.
• Facilitate engagement with the beneficiaries and other key stakeholders.
• Provide all necessary program documents and contacts of relevant stakeholders.
• Review the Consultant’s proposal, tools and evaluation report.

The consultant will:

• Be responsible for all aspects of the entire evaluation process, including evaluation preparation, data collection, analysis, and report writing.
• Be responsible for paying any tax or other fees related to this assignment.
• Provide all required working tools such as computer and data analysis software.

Reporting Arrangements

The Consultant will report to the IRC MEAL Coordinator and work in close collaboration with the partner management team, technical coordinators, and IRC leadership.

Duration of Assignment

The consultancy should not last more than 20 working days, excluding weekends.

Limitations and Mitigation Measures

1. Remote and low resource location: primary data collection will be conducted in a remote location; IRC will facilitate access to local enumerators who are already trained and experienced in data collection.

2. Unreliable internet connectivity: there is no mobile phone network, and internet is only available in offices. This may impact quantitative data collection and audio recording of qualitative data collection. Technology which enables offline collection and uploading at a later time should be used.

DELIVERABLES

1. Inception Report

The consultant is expected to develop an inception report upon commencement of the assignment. The inception report will detail the agreed methodologies to be employed during the evaluation. The report should also include the finalized activity plan and a structural outline of the final evaluation report and should be shared and approved by IRC before the data collection and analysis commencement.

The inception report should also contain:

• A detailed methodology including sampling for the evaluation implementation.
• A detailed schedule for the program review.
• The indicators that fall within the scope of the program review.
• Draft data collection tools (qualitative and quantitative). A work plan that sets out the preparatory activities, specific deliverables, and timeline related to the program review and budget for the data collection activities.

2. Facilitated Validation Session

The Consultant will facilitate a validation workshop preceding the delivery of a final report where the evaluation’s preliminary findings, conclusions, and recommendations will be presented to the IRC project team. The consultant will incorporate comments and feedback from the validation workshop into the final draft of the evaluation report

3. Evaluation Report

The report should address the above consultancy objectives and contain an executive summary, acknowledgments, introduction including program summary and purpose of the evaluation, a detailed methodology (including limitations), key findings (covering both document review and primary evaluation), lessons learned, evidence-based recommendations, conclusion and annexes. Annexes should include, at a minimum, field sites assessed and a list of key informants. A soft copy of the report should be shared with the MEAL Coordinator, and the report should not be more than 30 pages, excluding the cover page and annexes.

4. Summarized Four-Page Evaluation Report

Report that summarizes the evaluation purpose and background, evaluation questions, findings, lessons learned, conclusion and recommendations.

5. PowerPoint Presentation

Presentation of the evaluation process, key findings, lessons learned, and key recommendations.

The deliverables above will be complemented by regular communication and feedback from the IRC team and a validation workshop where the evaluation’s preliminary results will be presented to IRC leadership. After incorporating comments from the validation workshop, the report should be shared with the MEAL Coordinator, Grants Coordinator, and IRC senior management team.

Qualifications

REQUIREMENTS

Please note that this consultancy is only open to individual consultants and not companies or other evaluation firms.

• A qualification in Monitoring and Evaluation, Research, Statistics, Social Sciences, or any other related field is required.
• Knowledge and experience in conducting evaluations in low resource environments is required.
• Experience conducting evaluations in Sudan or in South Sudan or strongly preferred.
• Excellent skills in quantitative and qualitative data collection and analysis.
• Excellent skills in data analysis software such as SPSS or STATA (for quantitative data analysis) and ATLAS.ti, NVivo for qualitative data analysis.
• Experience in using mobile data collection systems such as Kobo Collect or Commcare will be an added advantage.
• Excellent organizing, facilitating, presentation, and communication skills, including report writing in English.
• Knowledge of Arabic is preferred.

APPLICATION PROCESS

Interested applicants who meet the required profile are invited to submit an application in English. The application package should include:

• The consultant CV with details of qualifications and relevant experience (maximum 5 pages)
• A technical proposal that summarizes an understanding of the ToR, methodology, and expected outputs. The Technical Proposal should have a complete description and explanation of the proposed methodology for the assignment, work plan, timeline, and any other resources that the consultant will make available to execute the task and achieve.
• Financial proposal showing the evaluation costs.

Applications that fail to include one or more of these elements will not be considered.

Payment rate and schedule will be determined between IRC and the selected consultant based on the financial proposal.