The African Population and Health Research Center (APHRC) seeks a consultant or small team to design, deploy on APHRC premises, and run an independent, reproducible evaluation of the FarajaMH generative AI screening model before and after fine-tuning. This consultancy aims to generate evidence for the Track A model freeze, evaluating factors such as screening accuracy, clinical reasoning, Swahili language understanding, safety, and cultural fit. Responsibilities include onboarding evaluation platforms on APHRC servers, validating test datasets, conducting blinded human-centred evaluations with clinicians and community members, and handing over a reusable evaluation harness. The consultant will report to the Principal Investigator.
Qualifications required include a PhD or Master's degree in machine learning, computer science, biostatistics, health data science, or a related field, and at least five years of experience in machine learning with a record of evaluating large language or speech models. Candidates should have expertise in AI for health, statistical skills (AUROC, calibration, fairness analysis), experience deploying containerized GPU workloads on-premises, and knowledge of research ethics and data protection laws in Kenya or Tanzania. Experience with Swahili, audio model evaluation, federated learning, and involvement of people with lived experience are advantageous.