What you'll be doing
- Owning the build of data source connectivity & transformation pattern templates — reusable by design
- Designing complex pipeline patterns for large and XL-complexity feeds, including CDC and parallel load
- Supporting the Lead Data Engineer on pattern design for correctness, performance and reusability ahead of feed supplier handover
- Working with a lakehouse-native stack on real, high-impact data problems
Tech stack
- AWS Glue
- S3/Iceberg
- MWAA
- Python
- PySpark
- Terraform
This is hands-on pipeline engineering on one of the most complex data feeds programmes in the public sector right now — not maintenance work, real build.
Rate:
£340/day
Location:
London
IR35 Status:
Inside
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Not Specified