Summary: The QA Specialist role focuses on ensuring the quality of AI-generated Java code evaluations through independent reviews and audits. The position requires strong Java expertise and involves collaborating with annotation teams to enhance evaluation accuracy and reliability. The role is fully remote, offering flexible hours and a milestone-based structure. Candidates should have extensive experience in Java quality assurance and a keen eye for detail.

Key Responsibilities:

• Conduct thorough quality assurance audits of Java code evaluations provided by annotators.
• Verify adherence to evaluation guidelines, ensuring annotators correctly identify issues related to functionality, executability, safety, performance, and documentation.
• Clearly document and communicate any discrepancies or errors found during evaluations, providing constructive and actionable feedback.
• Collaborate with annotation teams and stakeholders to refine standards and resolve quality-related concerns.
• Contribute to ongoing improvements in annotation quality, reliability, and efficiency.

Key Skills:

• 5–7+ years of professional experience in Java quality assurance, software testing, or technical evaluations.
• Strong expertise in Java programming, debugging, testing methodologies (unit and integration testing), and performance analysis.
• Exceptional analytical and problem-solving skills, with the ability to clearly articulate findings in written communication.
• English proficiency at B2, C1, C2, or Native levels.
• Prior experience with AI/ML training datasets, chatbot response evaluation, or data annotation verification (preferred).
• Familiarity with large language models (LLMs), reinforcement learning with human feedback (RLHF), or annotation platforms such as SuperAnnotate (preferred).
• Comfortable managing ambiguity and able to clarify complex technical requirements effectively.

Salary (Rate): undetermined

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT