About the Company
Senior Python Data Engineer | AWS | Spark | Airflow | Public Sector
Location: UK Remote / Hybrid
Rate: Competitive Daily Rate
Duration: Long-Term Contract
Sector: Public Sector
About the Role
This is an opportunity to work on large-scale data platforms, helping design, build and optimise modern cloud-native data services that support analytics, reporting and operational decision-making.
Responsibilities
- Design, develop and maintain scalable data pipelines and ETL/ELT solutions
- Build and optimise cloud-native data platforms within AWS
- Develop Spark and PySpark workloads for large-scale data processing
- Create and maintain Apache Airflow workflows and orchestration pipelines
- Implement Infrastructure as Code using Terraform
- Support batch and real-time data ingestion frameworks
- Work closely with data architects, analysts, data scientists and engineering teams
- Drive data quality, automation, monitoring and performance optimisation
- Contribute to CI/CD processes and engineering best practices
Qualifications Experience Required
- Previous experience delivering within public sector or highly regulated environments
- Experience working on large-scale cloud data transformation programmes
- Strong stakeholder engagement and problem-solving skills
- Ability to work independently and contribute within agile delivery teams
Required Skills
- Python
- AWS
- Apache Spark / PySpark
- Apache Airflow
- Terraform
- SQL
- ETL / ELT Development
- Data Integration
- Data Warehousing
- Cloud Data Platforms
Preferred Skills
- AWS Glue
- Athena
- EMR
- Lambda
- Databricks
- Kafka
- Docker
- Kubernetes
- GitLab CI/CD
- Jupyter Notebooks
- Snowflake
- Event-Driven Architectures
- Data Governance & Data Quality Frameworks
Equal Opportunity Statement
We welcome applications from all suitably qualified candidates and are committed to creating an inclusive environment for all employees, including those working within public sector or highly regulated environments.
Apply
If you have strong AWS, Python, Spark, Airflow and Terraform experience and would like to hear more about this opportunity, please apply with your latest CV and contact details.