Cloud Data Engineer - Cloud, Python

The Role

As a Data Engineer, you’ll help build and operate reliable, scalable data infrastructure and pipelines across an AWS cloud environment. Producing high-quality code in Python (and other languages, as needed).

Your responsibilities will include

• Design, build and maintain scalable data pipelines to ingest, transform and process large and complex datasets.
• Develop automated web scraping and bulk data acquisition workflows.
• Implement data transformation, validation, testing and quality controls to ensure datasets are reliable, reproducible and ready for analysis.
• Build and maintain containerised data and analytics applications, including dashboards and data tools.
• Implement and manage workflow orchestration and scheduling to ensure data pipelines and jobs are observable, reliable and maintainable.
• Apply Infrastructure as Code (IaC) to support data pipelines and processing workflows on AWS.
• Design and maintain systems for data version control and reproducible machine learning workflows (MLOps).
• Work closely with data scientists, analysts and engineers to translate analytical and operational requirements into effective data solutions.
• Contribute to a collaborative, multidisciplinary environment and share knowledge with colleagues.

About You

We’re looking for a Data or Software Engineer who has experience delivering production data solutions and enjoys solving complex data problems.

Essential experience

• Experience delivering production data pipelines end-to-end, including design, development, implementation and operational maintenance in a cloud environment.
• Experience writing production-quality Python to work with complex, messy or large datasets (Minimum 2 years)
• An undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development.
• A strong understanding of data engineering best practices, including data modelling, transformation, validation, testing.
• Experience working collaboratively within multidisciplinary teams.
• Strong communication skills, with the ability to explain technical work clearly and engage with non-technical audiences.

Why Join?

This is an opportunity to use your technical expertise on diverse projects that have a meaningful impact on the public good.

You’ll work alongside specialists from a range of disciplines, tackling interesting and complex data challenges while continuing to develop your technical skills.

You’ll also receive

• £49,400 salary + DDAT Allowance of up to £4,750 dependent upon experience
• Excellent pension
• 12-month fixed-term contract
• Hybrid working
• Choice of office location: London, Manchester, Cardiff, Belfast, Edinburgh or Darlington
• Opportunities for learning and professional development including 10 L & D days per annum.