AI Quality Engineer-Citizen

Job Summary

We are seeking an experienced AI Quality Engineer with strong expertise in Quality Assurance (QA), test automation, and AI-driven application validation. The ideal candidate will be responsible for validating AI-generated code, testing AI agents and AI-powered workflows, and ensuring software quality, security, reliability, and compliance throughout the development lifecycle.

The candidate should have experience working with CI/CD pipelines, automated testing frameworks, and technologies such as Salesforce, AWS, Java, and PostgreSQL, or comparable technology stacks.

Key Responsibilities

• Design, develop, and execute automated test strategies for AI-powered applications, AI agents, and traditional software systems.
• Validate AI-generated code for correctness, functionality, reliability, security, and maintainability.
• Test AI agents for response accuracy, consistency, reasoning behavior, tool usage, and workflow execution.
• Develop and maintain automated test scripts, test frameworks, regression suites, and integration tests.
• Implement quality gates within CI/CD pipelines to prevent defective or non-compliant code from reaching production.
• Perform functional, integration, API, performance, regression, and end-to-end testing.
• Evaluate AI outputs for hallucinations, unexpected behavior, edge cases, and potential security vulnerabilities.
• Validate application integrations involving Salesforce, AWS services, Java applications, PostgreSQL databases, or comparable technologies.
• Collaborate with developers, AI engineers, DevOps teams, and security stakeholders to identify and resolve defects.
• Ensure compliance with security standards, data protection requirements, and organizational quality policies.
• Document test results, defect reports, test coverage, and quality metrics.
• Continuously improve testing processes to support reliable AI-assisted software development.

Required Qualifications

• Strong experience in software QA, test automation, and software testing methodologies.
• Hands-on experience validating AI-generated code and AI-powered applications or agents.
• Experience developing automated test scripts using tools and frameworks such as Selenium, Playwright, Cypress, JUnit, TestNG, or PyTest.
• Understanding of AI/LLM testing, including output validation, hallucination detection, prompt testing, and evaluation of AI agent behavior.
• Experience integrating automated tests and quality gates into CI/CD pipelines.
• Knowledge of software security, compliance, vulnerability testing, and secure development practices.
• Experience with one or more of the following technologies:
• Salesforce platform and related integrations
• AWS cloud services
• Java application development and testing
• PostgreSQL, SQL, and database validation
• Strong analytical, debugging, documentation, and problem-solving skills.

Preferred Qualifications

• Experience with AI testing and evaluation frameworks such as DeepEval, Ragas, promptfoo, or similar tools.
• Familiarity with API testing tools such as Postman or REST Assured.
• Experience with Git, Jenkins, GitHub Actions, GitLab CI/CD, or comparable DevOps tools.
• Knowledge of SAST, DAST, software dependency scanning, and security testing.
• Experience testing RAG applications, LLM integrations, and agentic workflows.
• Understanding of responsible AI principles, data privacy, and AI governance.

Success Criteria

• Reliable and repeatable automated testing for AI-powered applications.
• Effective validation of AI-generated code and AI agent behavior.
• Strong test coverage and measurable quality improvements.
• Automated CI/CD quality gates that reduce defects and security risks.
• Compliance with applicable security and organizational standards.