Rubrics Generalist Expert (Remote)
Posted Today by Keystone Recruitment
Negotiable
Undetermined
Remote
EMEA
Summary: The Rubrics Generalist Expert role involves creating and evaluating rubrics for AI model performance in real-world shopping tasks. This position is remote and targets candidates with expertise in rubric design and evaluation, contributing to a project for a leading AI research lab. The role offers flexible scheduling and the potential for ongoing collaboration. Immediate start is available for selected candidates.
Key Responsibilities:
- Create and evaluate rubrics for user-generated shopping tasks
- Simulate realistic consumer behavior through prompt and scenario design
- Ensure AI outputs align with quality standards and task objectives
- Collaborate asynchronously with the AI research team to refine rubric coverage
Key Skills:
- Prior experience designing or using structured rubrics for data collection or evaluation
- Familiarity with AI training, evaluation, or prompt engineering tasks
- Excellent attention to detail and ability to assess nuanced task quality
- Strong written communication and organizational skills
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Title: Rubrics Generalist Expert (Remote)
Engagement: Hourly contract (independent contractor)
RATE: USD 40-45/Hour
Location: Remote – candidates based in the United States, United Kingdom, Canada, Europe, Singapore, Dubai, or Australia who are legally able to work as independent contractors in their jurisdiction.
Role Overview
One of our clients, a leading AI research lab, is developing a rubric dataset to evaluate AI model performance on real-world shopping tasks. We are seeking experts with experience in rubric creation and evaluation to contribute to this project. Your work will directly shape how AI systems understand and respond to nuanced consumer requests. This is a short-term project with immediate start, with potential for ongoing collaboration.
Key Responsibilities
- Create and evaluate rubrics for user-generated shopping tasks
- Simulate realistic consumer behavior through prompt and scenario design
- Ensure AI outputs align with quality standards and task objectives
- Collaborate asynchronously with the AI research team to refine rubric coverage
Ideal Qualifications
- Prior experience designing or using structured rubrics for data collection or evaluation
- Familiarity with AI training, evaluation, or prompt engineering tasks
- Excellent attention to detail and ability to assess nuanced task quality
- Strong written communication and organizational skills
Application Process
- Express interest by confirming availability and relevant experience
- Participate in a brief discussion or call to align on project expectations
- Selected candidates will begin work within 24 hours
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract and Payment Terms
- Engagement as an independent contractor
- Fully remote role with flexible scheduling
- Projects may be extended, shortened, or concluded early depending on needs and performance
- Work does not involve access to confidential or proprietary information from any employer, client, or institution
- Payments are weekly via Stripe or Wise based on hours worked
- H1-B or STEM OPT candidates cannot be supported at this time
APPLY NOW !