AI Data Engineer (contract)

Role Summary

This role is for a hands-on Data Engineer who will take full ownership of building and stabilising large-scale data pipelines using Microsoft Fabric and OneLake.

The focus is on transforming fragmented, high-volume data (billions of records) into clean, reliable, and analytics-ready datasets using medallion architecture.

The ideal candidate is proactive, solves data issues at the source, and ensures strong data quality, governance, and access control.

They will work closely with multiple teams, design resilient pipelines, handle frequent upstream changes, and enable trusted, self-service data for business users.

Key Responsibilities

• Build and maintain data pipelines using Microsoft Fabric + OneLake
• Implement medallion architecture (Bronze Silver Gold)
• Develop transformations using: SQL / T-SQL Power Query / M Query Notebooks (Python)
• Handle very large data volumes (billions of rows) efficiently
• Identify and fix data issues at the source
• Set up: Data quality checks Monitoring & alerting Logging frameworks
• Build semantic models (Power BI / Fabric) for analytics
• Implement data governance + access controls (RLS, object-level, etc.)
• Collaborate with: Upstream system owners Business/data analysts
• Continuously improve pipeline reliability and scalability

Must-Have Skills

• 4+ years in Data Engineering
• Hands-on experience with: Microsoft Fabric + OneLake
• Medallion architecture
• Strong experience in: SQL / T-SQL (advanced queries, optimisation)
• Data pipeline development (production-grade)
• Experience handling large-scale datasets (billions of rows)
• Strong in data transformation frameworks: Power Query / M Query Notebooks (Python preferred)
• Proven ability to: Troubleshoot data issues deeply Perform root-cause analysis
• Experience with: Data quality checks & validation frameworks Monitoring and alerting setup
• Hands-on with: Data governance Row-Level Security (RLS) / access controls
• Self-starter mindset (can work independently without direction)

Nice-to-Have Skills

• Microsoft Fabric certification
• Experience with: Power BI semantic modelling Data warehouse design (Fabric lakehouses / warehouses)
• Exposure to enterprise data governance frameworks
• Experience collaborating across multiple teams / systems

Experience Level Guidance

• 4–8 years total experience
• At least: 1–2 years in Microsoft Fabric ecosystem
• Background: Data Engineering (NOT Data Analyst heavy profiles)
• Must have worked on: Large-scale data environments Complex, messy, multi-source data systems

Additional Details

• Location: Remote (United Kingdom)
• Duration: 6 months
• Shift Timings: Monday – Friday (Business Hours)
• Pay Range: £303 - £341 /day (PAYE)
• Weekly Schedule: 40 hours