Lead Data Engineer
Up to £600 per day, Inside IR35
Initial 3-6 month contract
Remote first, with two days per quarter in Surrey office.
We are working with an established digital business looking for an experienced Lead Data Engineer to take technical ownership of its AWS-based data platform.
This is a hands-on leadership position. You will set the technical direction for the Data Engineering function while remaining close to delivery, designing solutions, reviewing architecture and code, improving platform reliability and mentoring the wider engineering team.
You will be joining during a period of significant investment and will play a key role in improving and evolving the organisation's data engineering capability.
Key responsibilities:
- Define the technical direction for an AWS-based data platform
- Design and deliver scalable, reliable and cost-effective data solutions
- Build and optimise batch and Real Time data pipelines
- Own and evolve the wider data architecture
- Establish engineering standards across code quality, testing and governance
- Improve data quality, observability, monitoring and platform reliability
- Drive the adoption of Infrastructure as Code, CI/CD and automation
- Develop robust data models supporting reporting, analytics, product insights and machine learning
- Lead technical delivery, incident resolution and root cause analysis
- Mentor Data Engineers and support the development of the wider team
- Work closely with analysts, data scientists and business stakeholders
We are looking for:
- Strong commercial Data Engineering experience, including technical leadership or mentoring
- Proven experience designing and building cloud-based data platforms on AWS
- Strong hands-on Python and SQL skills
- Production experience with dbt
- Experience with Amazon Redshift or a comparable cloud data warehouse
- Experience with Apache Airflow or similar orchestration tooling
- Infrastructure as Code experience, ideally Terraform
- Strong knowledge of CI/CD and modern software engineering practices
- Experience building production-grade batch and streaming data pipelines
- Strong understanding of data modelling, quality, observability and operational reliability
- The ability to communicate technical decisions clearly to both technical and non-technical stakeholders
This would suit a pragmatic Lead Data Engineer who is comfortable setting direction, making architectural decisions and remaining technically hands-on.
Please apply with your latest CV or contact me directly for further details.