Rubrics Generalist Expert (Remote)
Posted Today by Keystone Recruitment
£35 Per hour
Inside
Remote
EMEA
Summary: The Rubrics Generalist Expert role is a remote, hourly contract position focused on developing and evaluating rubric datasets for AI model performance in shopping tasks. Candidates will collaborate with an AI research team to ensure AI outputs meet quality standards. This short-term project offers immediate start opportunities and potential for ongoing collaboration. Ideal candidates will have experience in rubric creation and evaluation, particularly in the context of AI.
Key Responsibilities:
- Create and evaluate rubrics for user-generated shopping tasks
- Simulate realistic consumer behavior through prompt and scenario design
- Ensure AI outputs align with quality standards and task objectives
- Collaborate asynchronously with the AI research team to refine rubric coverage
Key Skills:
- Prior experience designing or using structured rubrics for data collection or evaluation
- Familiarity with AI training, evaluation, or prompt engineering tasks
- Excellent attention to detail and ability to assess nuanced task quality
- Strong written communication and organizational skills
Salary (Rate): £34.50 hourly
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: IT
Title: Rubrics Generalist Expert (Remote) Engagement: Hourly contract (independent contractor) RATE: USD 40-45/Hour Location: Remote – candidates based in the United States, United Kingdom, Canada, Europe, Singapore, Dubai, or Australia who are legally able to work as independent contractors in their jurisdiction.
Role Overview One of our clients, a leading AI research lab, is developing a rubric dataset to evaluate AI model performance on real-world shopping tasks. We are seeking experts with experience in rubric creation and evaluation to contribute to this project. Your work will directly shape how AI systems understand and respond to nuanced consumer requests. This is a short-term project with immediate start, with potential for ongoing collaboration.
Key Responsibilities
- Create and evaluate rubrics for user-generated shopping tasks
- Simulate realistic consumer behavior through prompt and scenario design
- Ensure AI outputs align with quality standards and task objectives
- Collaborate asynchronously with the AI research team to refine rubric coverage
Ideal Qualifications
- Prior experience designing or using structured rubrics for data collection or evaluation
- Familiarity with AI training, evaluation, or prompt engineering tasks
- Excellent attention to detail and ability to assess nuanced task quality
- Strong written communication and organizational skills
Application Process Express interest by confirming availability and relevant experience Participate in a brief discussion or call to align on project expectations Selected candidates will begin work within 24 hours We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract and Payment Terms Engagement as an independent contractor Fully remote role with flexible scheduling Projects may be extended, shortened, or concluded early depending on needs and performance Work does not involve access to confidential or proprietary information from any employer, client, or institution Payments are weekly via Stripe or Wise based on hours worked H1-B or STEM OPT candidates cannot be supported at this time APPLY NOW !