Key Responsibilities
- Design, build, and maintain scalable data pipelines on Microsoft Azure
- Develop reusable data products and frameworks to reduce duplication across teams
- Implement and maintain medallion architecture (Bronze, Silver, Gold layers) for structured data processing
- Work with cross-functional teams to promote shared standards, patterns, and engineering best practices
- Contribute to open and transparent development practices, including code sharing and documentation
- Develop and optimise data workflows using notebooks and distributed processing frameworks
- Ensure data quality, reliability, and performance across the platform
- Support the evolution of a modern enterprise data platform, reducing siloed solutions
Skills and Experiece Profile
- Strong experience with Python for data engineering
- Hands-on expertise in PySpark and distributed data processing
- Advanced SQL skills for data transformation and optimisation
- Experience developing in notebook environments (e.g., Fabric, Databricks, or similar)
- Practical experience with Microsoft Fabric or Azure data services
- Solid understanding of medallion architecture and layered data design
- Experience building and maintaining ETL/ELT pipelines
- Familiarity with version control (e.g., Git) and collaborative development workflows
Desirable Skills
- Experience working in large-scale enterprise or government environments
- Knowledge of data governance, security, and compliance practices
- Familiarity with CI/CD pipelines for data engineering
- Experience with data modelling and performance tuning
Additional Information
- Day Rate: Up to £550
- IR35 Status: Inside
- Start Date: November
- Travel: Fully remote working
- Required: SC Cleared
Rate:
£550/day
Location:
United Kingdom
IR35 Status:
Inside
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Not Specified