Negotiable
Undetermined
Remote
United Kingdom
Summary: The Finance Model Prompt Evaluator is responsible for creating and reviewing financial prompts that generate evaluable AI responses. This role requires a strong background in finance and the ability to craft prompts across various difficulty levels. The evaluator will collaborate with AI researchers to establish evaluation standards and ensure the quality of the prompts. The position is remote and requires a commitment of 10+ hours per week.
Key Responsibilities:
- Author clear, unambiguous, open-ended financial prompts that elicit evaluable AI responses
- Create original prompts across assigned subdomains at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional)
- Review authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness
- Ensure prompts require human judgment to evaluate AI response quality (e.g., quantitative analysis, risk modeling, regulatory reasoning)
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all prompts in a task are sufficiently distinct with varying difficulty levels
- Apply expert judgment to assess the depth and quality of financial reasoning required
- Edit prompts and difficulty assignments where standards are not met
- Collaborate with AI researchers to help establish rigorous evaluation standards for language models
Key Skills:
- Master’s degree or higher in Finance, Economics, Financial Engineering, or a closely related field
- Professional experience in financial services, investment banking, asset management, or a related field
- Strong command of financial modeling, quantitative methods, and domain-specific regulatory frameworks
- CFA, FRM, CPA, or equivalent professional certification preferred
- Excellent written English and ability to craft precise, well-scoped technical questions
Salary (Rate): £80.00/hr
City: undetermined
Country: United Kingdom
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: Other
Position: Finance Model Prompt Evaluator
Type: Hourly contract
Compensation: $50-$80 per hour
Location: Remote
Commitment: 10+ hours/week
Role Responsibilities
- Author clear, unambiguous, open-ended financial prompts that elicit evaluable AI responses
- Create original prompts across assigned subdomains at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional)
- Review authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness
- Ensure prompts require human judgment to evaluate AI response quality (e.g., quantitative analysis, risk modeling, regulatory reasoning)
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all prompts in a task are sufficiently distinct with varying difficulty levels
- Apply expert judgment to assess the depth and quality of financial reasoning required
- Edit prompts and difficulty assignments where standards are not met
- Collaborate with AI researchers to help establish rigorous evaluation standards for language models
Requirements
- Master’s degree or higher in Finance, Economics, Financial Engineering, or a closely related field
- Professional experience in financial services, investment banking, asset management, or a related field
- Strong command of financial modeling, quantitative methods, and domain-specific regulatory frameworks
- CFA, FRM, CPA, or equivalent professional certification preferred
- Excellent written English and ability to craft precise, well-scoped technical questions
Application Process (Takes 20 Mins)
- Upload resume
- Interview (15 min)
- Submit form