Senior Software Engineer, (OpenAI Codex, Claude Code or Cursor)

Accountabilities

• In this role, you will assess AI-generated coding interactions from the perspective of an experienced software engineer, focusing on technical judgment, clarity, usefulness, and the quality of the overall developer experience.
• Evaluate AI-generated coding interactions from beginning to end, including responses, explanations, and recommendations.
• Assess whether outputs are useful, broadly correct, and consistent with the judgment and practices of a strong software engineer.
• Review the quality of explanations and reasoning rather than focusing exclusively on code syntax or implementation details.
• Distinguish between different levels of response quality using consistent evaluation criteria.
• Provide clear, detailed, and constructive written feedback explaining what worked, what did not, and what felt inaccurate, misleading, or ineffective.
• Evaluate whether AI agents guide developers effectively, build trust, and provide meaningful assistance rather than simply generating output.
• Help define and refine quality standards for interactions with AI coding tools such as OpenAI Codex, Claude Code, and Cursor.
• Record short video explanations to communicate evaluation findings and provide additional context.

Requirements

• The ideal candidate is a senior software engineering professional with extensive experience, strong technical judgment, and a deep understanding of modern AI-assisted development workflows.
• You should be comfortable evaluating technical work critically, making subjective but well-reasoned assessments, and communicating your conclusions clearly.
• Senior, Staff, or Principal-level software engineer, or equivalent professional experience.
• Strong background in TypeScript, JavaScript, or Python.
• Hands-on experience using at least one of the following tools: OpenAI Codex, Claude Code, or Cursor.
• Deep familiarity with modern AI-assisted software development workflows.
• Ability to evaluate code and technical responses without necessarily executing the code or reviewing every line.
• Strong written and spoken English proficiency (B2 level or above), as detailed written feedback and short recorded video explanations are required.
• Confidence in providing direct, thoughtful, and opinionated feedback.
• High standards for software engineering quality and a strong understanding of what makes technical guidance genuinely useful.
• Ability to assess engineering judgment, communication quality, and developer experience beyond basic technical correctness.

Nice to have

• Previous experience with prompt design, AI evaluation, or model assessment workflows.
• Experience mentoring senior engineers or establishing engineering standards and best practices.

Benefits

• Competitive hourly compensation ranging from $100 to $200, depending on experience and location.
• Fully remote engagement, with worldwide opportunities subject to regional eligibility.
• Flexible workload of approximately 10–20 hours per week.
• Ongoing collaboration, with individual projects typically lasting from two weeks to several months.
• Opportunities to participate in additional projects based on evaluation performance.
• Flexible working hours.
• Streamlined selection process consisting of one take-home evaluation exercise and a recorded video walkthrough, with no interview required.
• Opportunity to contribute to the development and evaluation of cutting-edge AI coding tools.
• Contract-based engagement, with projects offered on an ongoing basis.
• Potential to start as soon as the evaluation exercise is successfully completed and a project position becomes available.