Data Platform Engineer/Data Streaming/Kinesis/Banking

Role Overview

We are looking for an experienced Data Platform Engineer with strong expertise in infrastructure, cloud platforms, containerisation and Real Time data streaming. The ideal candidate should have hands-on experience building, deploying and supporting scalable data platforms using Kafka, Kubernetes, Terraform, Amazon MSK and Amazon Kinesis.

Key Responsibilities

• Design, build and maintain scalable data platform infrastructure across cloud environments.
• Manage and support Apache Kafka/Amazon MSK clusters, including configuration, monitoring, scaling and performance optimisation.
• Work extensively with Amazon Kinesis and Real Time data streaming platforms.
• Deploy and manage data platform components using Kubernetes.
• Develop and maintain infrastructure using Terraform/Infrastructure as Code (IaC).
• Manage platform reliability, availability, scalability and performance.
• Implement monitoring, alerting, logging and troubleshooting for streaming and infrastructure components.
• Work with DevOps, Data Engineering and Cloud teams to build reliable data-platform solutions.
• Troubleshoot issues related to Kafka brokers, topics, partitions, replication, consumer groups and streaming pipelines.
• Implement security, access control and best practices across cloud and data-platform infrastructure.
• Automate platform provisioning, deployment and operational processes.

Mandatory Skills

• Strong experience as a Data Platform Engineer/Data Infrastructure Engineer/Platform Engineer.
• Hands-on experience with Apache Kafka.
• Strong experience with Amazon MSK (Managed Streaming for Apache Kafka).
• Good experience with Amazon Kinesis/Kinesis Data Streams.
• Strong Kubernetes experience.
• Strong Terraform/Infrastructure as Code experience.
• Good understanding of AWS infrastructure and cloud services.
• Experience with Real Time data streaming and distributed systems.
• Strong knowledge of monitoring, troubleshooting and performance optimisation.
• Experience working in production-scale environments.

Good to Have

• AWS certifications.
• Experience with Docker and CI/CD.
• Experience with Prometheus, Grafana, CloudWatch or similar monitoring tools.
• Experience with Helm and Kubernetes operators.
• Experience with Python, Java or Shell Scripting.
• Knowledge of networking, security, IAM and Linux administration.
• Experience with data lake/data platform architecture.

Key Focus

Infrastructure + Kafka + Kubernetes + Terraform OR Amazon MSK + Amazon Kinesis + Real Time Streaming