Grafana & Observability Engineer – AI Workflow Evaluation | Remote

Summary: The Grafana Expert (SWE) role involves designing and executing complex Grafana workflows, including dashboards and alerting systems, in a part-time capacity. The position requires strong technical skills in Grafana, PromQL, and Python, along with the ability to document workflows and validate task completion. The role is remote and demands a commitment of 10-15 hours per week. Candidates will also review AI-generated attempts and iterate on task designs to enhance evaluation quality.

Key Responsibilities:

• Design complex, multi-step Grafana workflows including dashboards, alerting, and data source configurations.
• Execute workflows to create reference solutions and validation paths.
• Write clear, structured task prompts with measurable outcomes.
• Develop programmatic graders to verify task completion.
• Review AI-generated attempts, identify failure points, and categorize issues.
• Iterate on task design to calibrate difficulty and improve evaluation quality.

Key Skills:

• Strong experience with Grafana in a professional environment.
• Strong experience with PromQL, dashboard templating, and alerting systems.
• Strong experience configuring data sources such as Prometheus and InfluxDB.
• Ability to clearly document workflows for programmatic validation.
• Strong experience with Python for writing basic grading scripts.
• Strong attention to detail and analytical thinking.
• Experience in SRE, platform engineering, or observability-related roles.

Salary (Rate): £150.00 hourly

City: undetermined

Country: United Kingdom

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT