AI Evaluator | $45/hr Remote

Summary: The role of Rubrics-Trained Generalist Expert involves creating and evaluating structured rubrics for user-generated shopping tasks in a remote setting. The position requires collaboration with AI research teams and a commitment of 10 to 40 hours per week. Candidates must have prior experience in rubric design and evaluation, along with strong communication skills. Attention to detail and the ability to work independently are essential for success in this role.

Key Responsibilities:

• Create and evaluate structured rubrics for user-generated shopping and consumer decision-making tasks.
• Design prompts that simulate realistic consumer behavior and shopping scenarios.
• Ensure rubric outputs meet defined quality standards and align with task objectives.
• Assess nuanced task performance and consistency using rubric-based evaluation methods.
• Collaborate asynchronously with AI research teams to refine rubric coverage and effectiveness.

Key Skills:

• Prior experience designing, applying, or evaluating structured rubrics.
• Familiarity with AI training, evaluation workflows, or prompt engineering tasks.
• Strong attention to detail and ability to evaluate nuanced task quality.
• Excellent written communication and organizational skills.
• Comfortable working independently in a remote, deliverables-driven environment.

Salary (Rate): £45.00/hr

City: undetermined

Country: United Kingdom

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: Other