Search Quality Analyst | $30/hr Remote

Summary: The Search Generalist role involves evaluating AI-generated search responses for quality and accuracy in a remote setting. Candidates will assess search queries, compare model responses, and contribute to structured evaluation workflows. Strong written English and research skills are essential, along with the ability to follow guidelines and provide feedback. Experience in search quality or related fields is preferred.

Key Responsibilities:

• Evaluate AI-generated search responses for factual accuracy, helpfulness, clarity, completeness, and overall quality.
• Assess whether models use search appropriately and whether search queries are well-formed and effective.
• Compare model responses side by side and provide concise, defensible rationales.
• Write and refine prompts, golden answers, rubric criteria, and edge cases for search-related evaluations.
• Apply project guidelines consistently across ambiguous, multi-step, and real-world search tasks.
• Identify recurring failure modes and escalate unclear cases or rubric gaps to project leads.
• Participate in calibration, QA, and feedback loops to maintain strong agreement and quality standards.
• Contribute to structured evaluation workflows to train, benchmark, and refine AI systems.

Key Skills:

• Excellent written English and strong online research skills.
• Strong judgment when synthesizing information from multiple sources.
• Ability to distinguish factual accuracy from fluency, confidence, or style.
• High attention to detail and comfort following structured guidelines.
• Reliable, self-directed, and responsive in an asynchronous remote environment.
• Experience in search quality, fact-checking, content evaluation, trust and safety, annotation, QA, or prompt/rubric writing preferred.
• Familiarity with search evaluation concepts such as factuality, helpfulness, severity, side-by-side comparisons, or tool-use assessment.
• Experience working with LLM evaluation workflows or human data projects.
• Multilingual skills are a plus.
• Bachelor’s degree preferred; advanced degree or strong professional background is a plus.

Salary (Rate): £30.00/hr

City: undetermined

Country: United Kingdom

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT