Terms of Reference (ToR) for the Evaluation of the Mission Alliance Project “Business for Development” (BfD) in Cambodia
1. Background
The Business for Development (BfD) project, launched in Cambodia in 2022, is an innovative initiative that aims to empower rural entrepreneurs through tailored business training, digital coaching, centred around the Roksi chatbot. Roksi serves as a virtual business advisor, providing accessible and practical tools for entrepreneurs to improve their skills, run profitable businesses, and build economic resilience. The Roksi service can be broken down into 3 main components; 1) the Roksi chatbot hosted on Telegram which users first interface with; 2) online training modules that the chatbot guides users to; 3) and access to individual coaching services provided to those who complete the online training modules. As the Roski service described here is the fundamental part of the BfD project, evaluating Roski is synonymous with evaluating the project.
The overarching goal of this evaluation is to understand the Roksi services’ effectiveness and efficiency in providing business support to rural entrepreneurs. Mission Alliance (MA) is also interested in assessing the Roksi services’ scalability and sustainability in Cambodia. As MA develops a new country program for Cambodia with a focus on economic inclusion and youth empowerment, this evaluation will also explore Roksi’s use case within the broader country program. In the context of this program, it is critical to evaluate how the Roksi service supports and integrates with Mission Alliance’s partners in Cambodia, ensuring alignment with their strategies and strengthening collaborative efforts to achieve shared goals in community development.
2. Mission Alliance Values and Principles
Mission Alliance (MA) is a value-based Christian organisation contributing to diaconal engagement and practicing Christian values in action. Our diaconal identity provides a joint foundation and language for partnering with churches and faith-based actors, who are central to upholding, shaping, and changing values, norms, and behaviour in communities.
MA’s mission statement is that “People are empowered to shape their own future.” Furthermore, the thematic goal of Economic Inclusion, of which this project is a part, states that “Individuals and businesses cultivate sustainable livelihoods and dignified work”.
Mission Alliance is committed to a culture of curiosity, exploration, innovation, and learning – seeking and documenting knowledge relevant to and needed in our work. MA has formulated guiding principles for generating new knowledge, when this is relevant. The principles are as follows:
• The knowledge is relevant to our organizational goals and mission.
• Knowledge is necessary and there is a plan for its use.
• The knowledge is not available elsewhere.
• Knowledge generation involving data collection from participants is done in an inclusive, dignified, and empowering way, and is aligned with Mission Alliance’s Ethical Guidelines.
3. Purpose of the Evaluation
The purpose of this evaluation is to provide a comprehensive assessment of the BfD project’s Roksi service, with a focus on understanding its relevance, coherence, effectiveness, efficiency, impact, and sustainability. These criteria will guide the evaluation as it seeks to address the following two areas of interest:
Evaluation and Learning: Assess the Roksi service based on how the current project has been implemented, identifying successes, challenges, and lessons learned.
Future Integration and Relevance: Explore how the Roksi service could potentially be integrated into and remain relevant within a future country program, particularly in alignment with MA partners strategic goals.
- Relevance:
- Is the Roksi service addressing the needs and priorities of its target users, particularly rural entrepreneurs in Cambodia?
- Do MA partners see the Roksi service as value added within their target areas?
- Coherence:
- How well does the Roksi service fit within the broader ecosystem of Mission Alliance’s programs and the development priorities in Cambodia?
- How does the Roksi service support and integrate with Mission Alliance’s partners in Cambodia, and to what extent does it align with their strategies and contribute to achieving their goals in community development?
- Effectiveness:
- Is the Roksi service achieving its intended objectives, such as improving users’ business skills, profitability, and economic resilience?
- What factors have contributed to or hindered the achievement of these objectives?
- How do the user interface, training modules, and coaching features contribute to the overall user experience and intended outcomes?
- Efficiency:
- How efficiently are resources being used to deliver the Roksi service, including training modules, coaching, and digital interfaces?
- What are the trade-offs between maximizing the number of users reached and fostering deeper synergies with local partners’ work on economic inclusion?
- What opportunities exist to optimize resource use while balancing the goals of broad outreach and strategic alignment with local partners?
- Regarding cost-efficiency how does Roksi’s coaching component compared to traditional face-to-face coaching approaches? What are the Roksi services unique advantages and limitations compared to other approaches within business development? Estimates of future cost-efficiency are also of interest, given recent measures to reduce costs in the project.
- Impact:
- What measurable changes has the Roksi service brought to its users and their communities?
- How have the experiences from the BfD project so far informed future opportunities for collaboration between Mission Alliance and local partners?
- Sustainability:
- What measures or strategies can strengthen the long-term viability and scalability of the intervention?
- What are the natural next steps for users after benefiting from the Roksi service?
By addressing these criteria, the evaluation aims to provide actionable recommendations and lessons for Mission Alliance’s future programming. This will enable strategic decision-making regarding the integration of Roksi into broader initiatives and its potential scalability to other contexts.
The findings will primarily inform Mission Alliance staff in Cambodia and the Oslo home office, while also generating valuable knowledge for external stakeholders and development practitioners.
4. Scope of the Evaluation
The evaluation will cover the period from January 2023 and up to the current implementation. Of the 4 original outcomes in the project design, this evaluation will be focusing on:
- Outcome 1: The effectiveness and impact of the Roksi service in enabling users to run businesses and achieve profitability.
- Focus on the 150 Roksi users who have received coaching services. However, the intention is to also sample the broader number of chatbot users who have not received coaching.
- The scope of the evaluation should also include an analysis of how the Roksi service supports and integrates with Mission Alliance’s partners in Cambodia. The evaluation should explore the potential for deepening synergies between Roksi and local partners’ economic inclusion initiatives, providing insights into future opportunities for programmatic integration and strategic alignment.
The other 3 outcomes from the original project design are no longer relevant to the project. This innovation project changed its strategic direction in 2023, rendering all but outcome 1 obsolete. It is therefore of little interest to evaluate the other 3 outcomes.
5. Methodology
The evaluation will employ a mixed-methods approach:
- Document Review: Read annual reports and the Roksi project brief.
- Secondary data: Analyze user data sets already obtained through Roksi.
- Surveys: Develop relevant survey questions with feedback from Mission Alliance staff, collect quantitative data from Roksi users to assess experiences and outcomes compared to baseline metrics and analyze the datasets.
- Focus Group Discussions: Develop relevant questions together with feedback from Mission Alliance staff, gather qualitative insights into user satisfaction, barriers, and areas for improvement to answer questions related to why and how, and qualitatively analyze the data.
- Interviews: Hold key stakeholder interviews with Mission Alliance partners who have collaborated with the BfD project, capturing partner perspectives on the relevance of the approach.
- Comparative Analysis: Evaluate how Roksi’s approach aligns with or diverges from industry standards for coaching and training. This includes benchmarking against best practices in digital and in-person models to assess the added value and limitations of the Roksi platform. Mission Alliance will support in finding a suitable comparable project.
Digital data collection and analysis tools are preferred.
6. Deliverables
- Inception report before data collection is initiated, including detailed outlining of evaluator’s understanding of the task, work plan, and methodology for data collection and analysis.
- A detailed evaluation report addressing all key questions, with actionable recommendations. Maximum 40 pages. A draft outline of the report should include an executive summary, introduction, methodology, findings, lessons learned, recommendations, conclusion, and annexes.
- The final evaluation report should be submitted by the 15th of May.
- A final presentation to Mission Alliance in Cambodia.
- All datasets must be submitted to Mission Alliance as CSV-files. The consultant is responsible for assuring anonymity of respondents in datasets.
7. Evaluator Qualifications
The evaluator(s) should have:
- Data Analysis Expertise: Ability to analyze quantitative and qualitative data to generate robust insights.
- Experience in Digital Service Evaluation: Familiarity with evaluating digital platforms or technology-based interventions, particularly in development contexts.
- Experience with entrepreneurship and MSMEs: The ideal evaluator has demonstrated expertise in entrepreneurship and working with MSMEs, particularly in rural or low-resource settings, and experience assessing digital tools for business development. Familiarity with the challenges faced by entrepreneurs and the ability to provide actionable recommendations for scalability and sustainability are essential.
How to apply
8. Submission of Proposals
Mission Alliance invites qualified consultants to submit a combined technical and financial proposal for the assignment. The technical proposal should outline your understanding of the Terms of Reference (ToR), proposed methodology, work plan, and relevant qualifications and experience. The financial proposal should provide a detailed budget, including all costs associated with delivering the assignment, expressed in USD. Proposals will be evaluated based on the quality and relevance of the technical proposal, as well as the cost-effectiveness of the financial proposal. Proposals must be submitted by February 3rd, 2025. Submissions can be sent to mos@nmakh.org and Jonathan.telfer@misjonsalliansen.no