Data Engineer - AWS - Python - SQL - SC Eligible

The Role:

Data Engineer - AWS - Python - SQL - SC Eligible requirement for an urgent start.

The Data Engineer - AWS - Python - SQL - SC Eligible will be working on a public sector AWS Data Platform.

• Ingesting data from relational databases, file transfers and APIs into an S3-based data lake
• Building ELT pipelines with dbt and Amazon Athena, using Apache Iceberg tables
• Writing and scheduling workflows in Apache Airflow
• Using Python for pre-processing, validation and automation, with tests in pytest
• Managing infrastructure as code with Terraform, and shipping through GitHub Actions CI/CD
• Building data quality checks so issues are caught before users see them
• Helping analysts and data scientists find, understand and use the data you deliver
• Keeping documentation current, so others can pick up your work

Data Engineer - AWS - Python - SQL - SC Eligible Key Skills:

• Commercial experience building production data pipelines
• Strong Python and SQL
• Hands-on AWS: S3, Athena, Glue, IAM & DMS
• dbt and Airflow, or close equivalents you can map across quickly
• Software engineering habits: git, code review, automated testing and CI/CD
• Able to explain trade-offs to non-technical colleagues
• Eligible for SC clearance

Nice to have:

• Terraform, Docker or Kubernetes
• Lambda or API Gateway for ingestion
• Spark or Glue PySpark for heavier workloads
• Dimensional modelling (star schemas, slowly changing dimensions)
• Experience in government or another highly regulated sector