AI Assurance Lead

The role

You'll take technical ownership of AI evaluation and assurance, working closely with engineering, product and data science teams.

Responsibilities will include:

• Designing model evaluation frameworks and test harnesses
• Evaluating and benchmarking LLMs against real-world use cases
• Measuring areas including precision, recall, grounding, calibration and hallucination
• Red teaming systems for prompt injection, jailbreaks, data leakage and tool misuse
• Stress-testing models using adversarial and out-of-distribution data
• Testing agent guardrails, permissions and containment
• Building automated evaluation into CI/CD pipelines
• Monitoring deployed models for drift and regression
• Supporting fine-tuning and model adaptation
• Producing technical assurance evidence and presenting findings to senior stakeholders and customers

What we're looking for

You'll bring a strong Machine Learning Engineering or Data Science background alongside hands-on experience taking AI systems into production.

Key experience includes:

• Production Python
• PyTorch, Hugging Face Transformers or similar
• Strong hands-on LLM experience
• RAG, vector search, prompting, tool use and agent orchestration
• Designing and running model evaluations
• Cloud deployment, containers and CI/CD
• Understanding AI attack surfaces and failure modes
• Applied statistics and experimental design

Experience in AI safety, adversarial ML, AI red teaming, technical assurance or Defence/Secure Government environments would be highly advantageous.

A rare opportunity to combine hands-on AI engineering with safety and assurance, working on technically challenging systems where reliability and trust genuinely matter.