Urdu LLM Evaluator
Posted Today by Himalayas
Negotiable
Outside
Remote
Remote
Summary: The role of Urdu LLM Evaluator involves reviewing and assessing model-generated replies in Urdu, focusing on either Tone or Fluency. Evaluators will provide feedback on the quality and naturalness of responses, contributing to the validation of evaluation frameworks for AI development. This position is project-based and allows for remote work as an Independent Contractor with CrowdGen. Strong fluency in Urdu and comprehension of English are essential for success in this role.
Key Responsibilities:
- Evaluate model replies in your native language based on either Tone or Fluency.
- Assess the overall quality, correctness, and naturalness of responses.
- Read the user prompt and two model replies, then rate each using a five-point scale.
- Provide brief rationales for any extreme ratings.
Key Skills:
- Native-level fluency in Urdu.
- Strong comprehension of English.
- Ability to assess quality dimensions such as Tone and Fluency.
- Experience with evaluating language model outputs.
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: Other