Summary: We are seeking a highly experienced and skilled Senior Kafka Engineer to join our team, focusing on designing, implementing, and maintaining large-scale Kafka deployments. The ideal candidate will have a proven track record in upgrading Kafka clusters and a deep understanding of Kafka's architecture and best practices. This role requires both independent and collaborative work in a fast-paced environment.

Key Responsibilities:

• Design, implement, and maintain high-throughput, low-latency Kafka clusters.
• Lead the upgrade of existing Kafka clusters from version 3.3.1 to the latest supported version, ensuring minimal downtime and data integrity.
• Experience in Confluent migration.
• Develop and implement monitoring and alerting systems for Kafka.
• Troubleshoot and resolve Kafka-related issues.
• Participate in on-call rotation to ensure 24/7 system availability.
• Collaborate with other engineers to integrate Kafka into existing systems.
• Design and implement Kafka security best practices.
• Contribute to the improvement of our Kafka infrastructure.
• Mentor junior engineers and share expertise.
• Stay current with the latest Kafka technologies and best practices.

Key Skills:

• 8+ years of experience working with Apache Kafka.
• Proven experience upgrading Kafka clusters, specifically from version 3.3.1 to a current supported version.
• Strong understanding of Kafka architecture, including topics, partitions, consumers, and producers.
• Experience with Kafka Connect.
• Experience with Kafka Zookeeper, Kraft.
• Experience working in Confluent Kafka.
• Experience in deploying the Kafka image in Kubernetes.
• Experience in setting up Stretched Kafka Cluster.
• Experience with Kafka Streams or ksqlDB a plus.
• Proficiency in at least one scripting language (e.g., Python, Shell).
• Experience with cloud-based Kafka deployments (AWS, Azure, GCP) is highly preferred.
• Experience with Kafka monitoring and alerting tools.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Experience with schema registries (e.g., Avro, Confluent Schema Registry) is a bonus.
• Contributions to open-source Kafka projects is a bonus.
• Relevant certifications (e.g., Confluent Certified Kafka Developer) is a bonus.

Salary (Rate): undetermined

City: Sheffield

Country: United Kingdom

Working Arrangements: hybrid

IR35 Status: undetermined

Seniority Level: Senior

Industry: IT