Software Engineer (Code QA) - Remote
Posted Today by Keystone Recruitment
£84 Per hour
Undetermined
Remote
EMEA
Summary: The Software Engineer (Code QA) role is a remote position focused on supporting AI research by creating evaluation datasets that assess AI's reasoning and explanation quality in coding contexts. Candidates will craft realistic developer prompts and write technically accurate responses, simulating authentic developer interactions. This position is ideal for engineers with strong technical writing skills and a critical approach to software development. The role offers part-time flexibility and is suitable for those with a background in software engineering or technical education.
Key Responsibilities:
- Craft realistic developer prompts across categories such as code review, debugging, error diagnosis, configuration issues, and refactoring.
- Source and adapt content from real pull requests to create authentic technical scenarios.
- Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality.
- Ensure responses reflect sound engineering judgment and best practices.
Key Skills:
- 2+ years of experience in software engineering, technical research, or technical education/content development.
- Bachelor’s degree in Computer Science, Software Engineering, or related field (advanced degree preferred).
- Strong proficiency in one or more languages such as Python, JavaScript, Java, or C++.
- Hands-on experience with debugging, testing, and code validation.
- Strong technical writing skills and attention to detail.
- Ability to clearly articulate trade-offs, assumptions, and reasoning.
Salary (Rate): £84 hourly
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Role: Software Engineer (Code QA) - Remote
Location: 100% Remote (Global)
Compensation: USD 70-120 per hour
Role Overview
One of our clients is hiring experienced Software Engineers to support high-impact research collaborations with leading AI labs. In this role, you will contribute to building evaluation datasets designed to assess AI reasoning, explanation quality, and technical judgment in coding-related interactions. This opportunity is ideal for engineers who enjoy thinking critically about code, articulating technical reasoning clearly, and shaping how AI systems understand software development workflows.
About the Project
The project focuses on code-based question-and-answer datasets that evaluate natural-language reasoning and technical explanation quality — rather than simply executable correctness. Tasks are structured as realistic, chat-style developer prompts that include all relevant context inline, such as:
- Code snippets
- Error messages
- Logs
- Configuration files
- Feature requirements
The goal is to simulate authentic developer interactions and evaluate how well AI systems reason through them.
Key Responsibilities
- Craft realistic developer prompts across categories such as: Code review, Debugging, Error diagnosis, Configuration issues, Refactoring and design trade-offs
- Source and adapt content from real pull requests to create authentic technical scenarios
- Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality
- Ensure responses reflect sound engineering judgment and best practices
Ideal Qualifications
- 2+ years of experience in software engineering, technical research, or technical education/content development
- Bachelor’s degree in Computer Science, Software Engineering, or related field (advanced degree preferred)
- Strong proficiency in one or more languages such as Python, JavaScript, Java, or C++
- Hands-on experience with debugging, testing, and code validation
- Strong technical writing skills and attention to detail
- Ability to clearly articulate trade-offs, assumptions, and reasoning
Project Details
Start Date: Immediate
Duration: 1–2 months
Commitment: Part-time (15–25 hours per week, with flexibility up to 40 hours per week)
Fully remote and asynchronous
Application & Onboarding Process
- Submit your resume
- Complete a short AI-led interview (approximately 15 minutes)
- Receive follow-up communication within a few days regarding next steps and onboarding
Compensation & Contract Terms
- $70–$120 per hour, depending on experience and expertise
- Weekly payments via supported payment platforms
- Independent contractor classification
- Project scope may be extended, shortened, or concluded based on performance and business needs
- No access to confidential or proprietary information from any employer, client, or institution
APPLY NOW!