AI Content Quality Analyst (Remote)
Posted 1 day ago by Taskify AI
£36 Per hour
Undetermined
Remote
EMEA
Summary: The AI Content Quality Analyst role is a remote, part-time independent contractor position focused on evaluating and improving AI-generated content. The analyst will assess the quality of AI outputs, identify errors, and provide structured feedback to enhance model performance. This position requires strong analytical skills and the ability to work independently across various projects. Candidates must be legally authorized to work in their respective countries, including the US, UK, Canada, Ireland, Australia, and New Zealand.
Key Responsibilities:
- Analyze large sets of AI-generated responses to identify trends in model behavior, such as repetitive phrasing, logical inconsistencies, or refusal to follow constraints.
- Perform detailed quality assessments using complex rubrics, rating content on multiple dimensions (e.g., coherence, truthfulness, safety).
- Edit and rewrite subpar AI responses to create "Gold Standard" examples that demonstrate exactly how the model should have replied.
- Flag and categorize safety violations, including bias, hate speech, or dangerous content, ensuring strict adherence to responsible AI policies.
- Provide feedback on the annotation guidelines themselves, identifying ambiguities or gaps in logic that may be causing inconsistent data labeling.
Key Skills:
- Proven experience in Quality Assurance (QA), Content Analysis, Data Annotation, or Editing.
- Strong analytical skills with the ability to look at text objectively and diagnose why a response failed (e.g., failed logic vs. failed grammar).
- Excellent command of English with the ability to spot subtle nuances in tone and meaning.
- Ability to internalize and apply complex, evolving rule sets and decision trees without deviation.
- Comfortable working independently in a remote environment, managing time across multiple projects.
Salary (Rate): £36.00 hourly
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Job Title: AI Content Quality Analyst (Remote)
Employment Type: Remote, part-time independent contractor
Location: Remote within United States, United Kingdom, Canada, Ireland, Australia & New Zealand (You must be legally authorised to work in the country where you are based)
Compensation: $35 - $45 per hour based on location, experience, responsibilities, and performance.
Role Overview: We are hiring on behalf of one of our clients, a leading global player in the AI space. This remote, part-time contract role places you in a critical diagnostic position within the AI training loop. As a Content Quality Analyst, you will not only review individual pieces of text but also analyze the broader performance of the model. You will assess AI outputs against rigorous quality standards, identify recurring patterns of error, and provide the structured data needed to engineer a more accurate, helpful, and safe AI.
Key Responsibilities:
- Analyze large sets of AI-generated responses to identify trends in model behavior, such as repetitive phrasing, logical inconsistencies, or refusal to follow constraints.
- Perform detailed quality assessments using complex rubrics, rating content on multiple dimensions (e.g., coherence, truthfulness, safety).
- Edit and rewrite subpar AI responses to create "Gold Standard" examples that demonstrate exactly how the model should have replied.
- Flag and categorize safety violations, including bias, hate speech, or dangerous content, ensuring strict adherence to responsible AI policies.
- Provide feedback on the annotation guidelines themselves, identifying ambiguities or gaps in logic that may be causing inconsistent data labeling.
Qualifications:
- Proven experience in Quality Assurance (QA), Content Analysis, Data Annotation, or Editing.
- Strong analytical skills with the ability to look at text objectively and diagnose why a response failed (e.g., failed logic vs. failed grammar).
- Excellent command of English with the ability to spot subtle nuances in tone and meaning.
- Ability to internalize and apply complex, evolving rule sets and decision trees without deviation.
- Comfortable working independently in a remote environment, managing time across multiple projects.
Compensation and Benefits:
Rate: Earn up to $45 USD/hr (Rates vary based on project complexity and expertise).
Flexibility: 100% remote with no fixed schedule. Contributors typically spend 5–10 hours per week, with the option to work up to 40 hours.
Equal Opportunity Employer: We're committed to fostering a diverse and inclusive work environment. We welcome applicants from all backgrounds and celebrate diversity in our workforce.