Data Labeling Analyst III

We are seeking detail-oriented and skilled contractors to support labeling and quality assurance for Large Language Model (LLM) training focused on Premier Partner Support. The successful candidates will play a critical role in training AI models to detect and understand policy guidelines, with a particular emphasis on account name changes. Your work will ensure high-quality labeling accuracy and consistency, directly impacting the effectiveness of our AI-driven protection systems.

Key Responsibilities

• Perform detailed labeling and annotation of data related to Premier Partner Support, including account change approvals, denials, policy exceptions, and related abuse, to support LLM training
• Ensure accuracy, consistency, and strict adherence to labeling guidelines and policies.
• Collaborate with internal teams to create and maintain golden sets for quality control and model calibration.
• Identify and report patterns or trends in support requests related to account changes that may indicate policy violations or require exceptions, to inform model improvements.
• Participate in vendor site calibration and quality assurance processes as needed.
• Provide actionable feedback to tooling and engineering teams to improve labeling workflows and annotation tools.
• Communicate effectively with internal stakeholders to align on labeling standards and quality expectations.
• Contribute to process improvements and documentation related to labeling workflows.
• Stay up-to-date with policy changes, labeling guidelines, and industry best practices related to account changes.

Minimum Qualifications

• Experience in data labeling, quality assurance, or related fields (minimum 4+ years).
• Direct experience working in operational support, combined with a strong understanding of policy guidelines is highly relevant.
• Excellent communication skill and perfect English with writing and speaking.
• Strong attention to detail and ability to maintain high accuracy in a fast-paced environment.
• Ability to work independently and collaboratively within a team.
• Familiarity with platforms, policy enforcement, or integrity problems is a plus.
• Knowledge of social media platforms, account structures, and enforcement policies.

Preferred Qualifications

• Experience with AI/ML model training or data annotation for machine learning.
• Technical skills (e.g., experience with annotation tools, providing feedback to engineering teams, or basic scripting) to support tooling improvements.
• Prior experience working with vendor management or quality calibration.
• Certification or training in content moderation or data labeling.

Working Conditions

• This is a contractor role with an in office setting.
• Must be able to work in a fast-paced environment with multiple priorities and deadlines.

Years of Experience

• 3-4 years of experience as a data labeling, quality assurance, or related fields

Degrees/Certifications Required

• Bachelor s degree nice to have

Top 3 Must-have HARD skills

• Strong attention to detail and ability to maintain high accuracy in a fast-paced environment
• Direct experience working in operational support, combined with a strong understanding of policy guidelines is highly relevant
• Analytical mindset with the ability to identify patterns and anomalies in complex data

Good to have skills

• Analytical mindset with the ability to identify patterns and anomalies in complex data
• Certification or training in content moderation or data labeling
• Experience with AI/ML model training or data annotation for machine learning.

Interview

• 1 round