AI Test engineer

AI Test engineer

Posted 1 day ago by Opus Recruitment Solutions Ltd

Apply
£250 Per day
Outside
Undetermined
Cambridge, UK
Apply
.NET Framework Agile Methodology Apache JMeter Application Programming Interface (API) Artificial Intelligence Azure DevOps Azure Kubernetes Service Cloud-Native Applications Continuous Integration and Continuous Delivery DevOps Input/Output Kubernetes Microsoft 365 Microsoft Azure Open Web Application Security Project (OWASP) Penetration Testing Profiling (Computer Programming) Risk Management Framework Scrum (Software Development) Security Testing Selenium (Software) Software Testing Test Engineering Test Suite Threat Modeling Verification And Validation

Summary: The AI Test Engineer role involves ensuring the reliability, security, and performance of distributed, cloud-native services and public APIs for enterprise-scale AI systems. The position requires collaboration within a fast-paced Scrum team to develop automated testing frameworks and validation strategies. Candidates should have extensive experience in software testing, particularly in cloud-native applications, and a strong understanding of AI systems. This is a contract position with a high potential for extension beyond the initial term.

Key Responsibilities:

  • Test and validate cloud-native services deployed on Microsoft Azure.
  • Build and maintain automated test suites for APIs, service components, and AI pipelines.
  • Automate evaluation of AI outputs for accuracy, safety, and consistency.
  • Define quality metrics and observability hooks in collaboration with engineering teams.
  • Ensure compliance with AI regulations and standards such as NIST AI RMF and the EU AI Act.
  • Conduct threat modelling and security testing for APIs and AI services.
  • Provide early feedback to improve design and reduce defects.
  • Mentor junior engineers and contribute to a culture of innovation and continuous improvement.

Key Skills:

  • 12+ years in software testing and validation for cloud-native applications using Microsoft and .NET.
  • Expertise in automated testing frameworks such as Selenium, Postman, JMeter, or custom-built solutions.
  • Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized environments (Docker, Kubernetes).
  • Strong knowledge of API testing, performance profiling, and security practices including OWASP Top 10 and penetration testing.
  • Familiarity with AI/ML systems, including LLM evaluation techniques, output scoring, and validation frameworks.
  • Understanding of prompt engineering, RAG, model orchestration, and hallucination detection.
  • Awareness of ethical AI, bias detection, explainability, and regulatory compliance.
  • Proven ability to work in Agile teams, solve complex problems, and mentor junior team members.

Salary (Rate): £250 per day

City: Cambridge

Country: UK

Working Arrangements: undetermined

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

AI Test Engineer
Day Rate: £200-£250 per day (Outside IR35)
Location: Cambridge
Contract: 6-month initial contract - high possibility for 18 months.

About the Role
Our client is hiring an experienced AI Test Engineer to join a prestigious organisation as part of our Core AI Services team. This role is focused on ensuring the reliability, security, and performance of distributed, cloud-native services and public APIs that underpin enterprise-scale AI systems. You'll work in a fast-paced Scrum team alongside developers, architects, and data scientists to build automated testing frameworks, AI evaluation tools, and validation strategies. If you thrive on experimentation, problem-solving, and building robust systems that scale, this is the opportunity for you.

Key Responsibilities

  • Test and validate cloud-native services deployed on Microsoft Azure.
  • Build and maintain automated test suites for APIs, service components, and AI pipelines.
  • Automate evaluation of AI outputs for accuracy, safety, and consistency.
  • Define quality metrics and observability hooks in collaboration with engineering teams.
  • Ensure compliance with AI regulations and standards such as NIST AI RMF and the EU AI Act.
  • Conduct threat modelling and security testing for APIs and AI services.
  • Provide early feedback to improve design and reduce defects.
  • Mentor junior engineers and contribute to a culture of innovation and continuous improvement.

Skills & Experience

  • 12+ years in software testing and validation for cloud-native applications using Microsoft and .NET.
  • Expertise in automated testing frameworks such as Selenium, Postman, JMeter, or custom-built solutions.
  • Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized environments (Docker, Kubernetes).
  • Strong knowledge of API testing, performance profiling, and security practices including OWASP Top 10 and penetration testing.
  • Familiarity with AI/ML systems, including LLM evaluation techniques, output scoring, and validation frameworks.
  • Understanding of prompt engineering, RAG, model orchestration, and hallucination detection.
  • Awareness of ethical AI, bias detection, explainability, and regulatory compliance.
  • Proven ability to work in Agile teams, solve complex problems, and mentor junior team members.

What You'll Gain

  • Join a team building cutting-edge AI systems at enterprise scale.
  • Collaborate in a global R&D environment with experts across domains.
  • Contribute to the development of ethical, trustworthy, and future-ready AI services.
  • Enjoy competitive day rates with a contract Outside IR35.

Apply
Similar Jobs
Loading data...