Summary: The role of Senior Data Engineer (Infrastructure) involves designing, building, and maintaining complex data infrastructure systems in both on-premises and Azure cloud environments. This position requires expertise in developing data pipelines, managing hybrid cloud solutions, and ensuring data security and compliance. The engineer will play a crucial role in supporting advanced analytics and data-driven decision-making within the organization. Candidates must hold SC clearance, with DV preferred, and the position is fully onsite near Worcester/Great Malvern/Gloucester.

Key Responsibilities:

• Develop and manage data pipelines using Python for streaming and batch data processing.
• Implement and manage hybrid cloud and data storage solutions across on-premises and Azure.
• Utilise Docker for containerisation and Kubernetes for orchestration of applications.
• Automate data workflows using tools like Apache NiFi and Apache Airflow.
• Handle real-time data streams with event-driven technologies such as Kafka.
• Manage security setups and access controls to ensure data integrity.
• Enhance Elasticsearch setups for data searching and analytics capabilities.
• Administer and optimise PostgreSQL databases for performance and availability.

Key Skills:

• Strong Python skills for scripting and automating data processes.
• Experience with ETL/ELT workflows in hybrid environments.
• Expertise in hybrid cloud data architecture integrating on-premises and Azure.
• Solid experience with Docker and Kubernetes for application management.
• Proficiency in workflow automation tools like Apache NiFi and Apache Airflow.
• Experience in managing event streaming platforms like Kafka.
• Knowledge of data security practices and tools, including Keycloak.
• Strong background in managing Elasticsearch and PostgreSQL.

Salary (Rate): 700

City: Worcester

Country: United Kingdom

Working Arrangements: on-site

IR35 Status: outside IR35

Seniority Level: Senior

Industry: IT