ROLE OVERVIEW
Build and optimise data pipelines using Java (preferred) or Python.
Develop real-time streaming and batch data pipelines.
Work with Kafka, Flink or Spark, with Kafka/Flink preferred.
Develop AWS-based data solutions using Kinesis, MSK, Lambda and Glue.
Implement data governance, lineage, security and quality controls.
Support CI/CD using Azure DevOps, GitHub Actions or Jenkins.
Use Terraform and Infrastructure as Code practices.
Deploy containerised workloads using Docker, Kubernetes and AWS EKS.
Collaborate with analytics, AI/ML, product and engineering teams.