Python Senior Software Engineer, AI Training

Accountabilities

• In this role, you will evaluate AI-generated coding interactions and provide expert feedback to help improve the quality, reliability, and usefulness of AI-powered development tools.
• Evaluate AI-generated coding interactions from beginning to end, assessing their overall quality and effectiveness.
• Determine whether AI outputs are useful, technically sound at a high level, and aligned with the reasoning and practices of an experienced software engineer.
• Assess the quality of explanations, reasoning, and recommendations rather than focusing exclusively on code correctness.
• Differentiate between varying levels of response quality and apply consistent evaluation standards.
• Provide clear, detailed, and opinionated written feedback on what works, what does not, and what may feel inaccurate, misleading, or ineffective.
• Assess whether AI models guide developers effectively, provide meaningful context, and build trust through their interactions.
• Help define and refine quality benchmarks for AI coding agents, including tools such as OpenAI Codex, Claude Code, and Cursor.
• Record short video explanations to communicate evaluation findings and provide additional context.

Requirements

• The ideal candidate is a highly experienced software engineer with strong Python expertise, a deep understanding of modern AI-assisted development workflows, and the ability to make rigorous technical assessments.
• Senior, Staff, or Principal-level software engineering experience, or equivalent professional expertise.
• Strong professional background in Python, with additional experience in TypeScript or JavaScript also relevant.
• Hands-on experience with at least one of the following AI coding tools: OpenAI Codex, Claude Code, or Cursor.
• Deep familiarity with modern AI-assisted software development workflows.
• Ability to evaluate code and technical responses without necessarily executing the code or reviewing every line.
• Strong written and spoken English proficiency (B2 level or above), as the role involves detailed written feedback and recorded video explanations.
• Confidence in providing direct, constructive, and well-supported opinions.
• Excellent critical thinking and communication skills, with a high standard for engineering quality.
• Ability to distinguish between technical correctness and genuinely useful engineering guidance.

Nice to have

• Previous exposure to prompt design, AI evaluation, or model assessment workflows.
• Experience mentoring senior engineers or defining engineering standards and best practices.

Benefits

• Competitive hourly compensation ranging from $100 to $200, depending on experience and location.
• Fully remote contract opportunity, with worldwide availability subject to regional eligibility.
• Flexible workload of approximately 10–20 hours per week.
• Flexible working hours.
• Ongoing engagement, with individual projects typically lasting from two weeks to several months.
• Opportunities to participate in additional projects based on evaluation performance.
• Potential to start as soon as the take-home assessment is completed successfully and a project position becomes available.
• Streamlined selection process consisting of one take-home evaluation exercise and a recorded video walkthrough, with no interview required.
• Opportunity to contribute to the development and evaluation of advanced AI coding technologies.