 Hays.png)
QA Engineer
Posted 3 days ago by Hays
Negotiable
Inside
Hybrid
London
Summary: The QA Engineer role focuses on testing AI-powered products, specifically LLM-based applications and agentic AI systems. The position involves building evaluation frameworks and ensuring quality throughout the development life cycle. The role requires a strong background in Quality Engineering and hands-on experience with AI technologies. This is a 6-month contract with a hybrid working arrangement in London.
Key Responsibilities:
- Testing LLM-based applications and agentic AI systems.
- Building robust evaluation frameworks.
- Driving quality across the entire development life cycle.
Key Skills:
- Strong background in Quality Engineering, Test Automation, and QA best practices.
- Hands-on experience testing LLMs, Generative AI applications, and AI agents.
- Experience with golden datasets, AI evaluation frameworks, and LLM-as-a-Judge methodologies.
- Strong Python skills and software engineering fundamentals.
- Experience building test automation, evaluation, or CI/CD pipelines.
- Knowledge of AI-specific challenges including hallucinations, reliability, prompt evaluation, safety, and agent orchestration.
- A curious, forward-thinking mindset with a passion for staying on top of the latest AI and testing innovations.
Salary (Rate): £699 inside IR35
City: London
Country: United Kingdom
Working Arrangements: hybrid
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: IT
QA Engineer – AI & Agentic Systems
We're looking for a QA Engineer to help shape the next generation of AI-powered products. You'll be responsible for testing LLM-based applications and agentic AI systems, building robust evaluation frameworks, and driving quality across the entire development life cycle.
What You'll Bring
Strong background in Quality Engineering, Test Automation, and QA best practices.
Hands-on experience testing LLMs, Generative AI applications, and AI agents.
Experience with golden datasets, AI evaluation frameworks, and LLM-as-a-Judge methodologies.
Strong Python skills and software engineering fundamentals.
Experience building test automation, evaluation, or CI/CD pipelines.
Knowledge of AI-specific challenges including hallucinations, reliability, prompt evaluation, safety, and agent orchestration.
A curious, forward-thinking mindset with a passion for staying on top of the latest AI and testing innovations.
Nice to Have
LLM evaluation experience (golden datasets, LLM-as-a-judge, automated evaluation frameworks)
Quality Engineering mindset (metrics, strategy, automation, testing best practices)
Experience with Agentic AI, MCP tools, and modern AI testing frameworks.
Strong Python and engineering capability
Hybrid working: 2 days per week in the London office.
6-month contract
£699 inside IR35
If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us now.
Email Abdiaziz.Ahmed@hays.com with an updated CV if you believe you are the right fit.
If this job isn't quite right for you, but you are looking for a new position, please contact us for a confidential discussion about your career.