Data Engineer - Infrastructure & Data Engineering

We are looking for an experienced Data Engineer to join a team responsible for designing, building and operating a modern Azure Data Platform. The role combines Azure infrastructure, Databricks platform engineering, Terraform and data engineering practices, with a strong focus on automation, reliability and self-service capabilities.

Key Responsibilities

• Design, build and maintain CI/CD pipelines using Azure DevOps, GitHub Actions or similar tools.
• Develop and maintain Terraform Infrastructure as Code (IaC) for Azure services and Azure Databricks.
• Support and manage Azure Databricks, including Unity Catalog, workspaces, clusters and jobs.
• Develop Python-based automation and platform tooling.
• Build reusable modules and libraries to support data engineers and platform users.
• Support platform governance, monitoring, provisioning and cost management.
• Troubleshoot infrastructure, connectivity, authentication and pipeline issues.
• Work closely with Data Engineering, Security and Infrastructure teams to deliver platform capabilities.
• Maintain technical documentation and contribute to platform standards and best practices.

Essential Skills & Experience

• Strong hands-on experience with Microsoft Azure infrastructure, including Entra ID, Key Vault, networking and storage.
• Hands-on Azure Databricks experience, particularly Unity Catalog, workspace, cluster and job management.
• Strong Terraform/Infrastructure as Code experience, ideally across multiple environments.
• Strong Python development experience for automation and platform tooling.
• Experience building and maintaining CI/CD pipelines using Azure DevOps, GitHub Actions or similar.
• Good understanding of data platform governance and cost management.
• Comfortable working across both cloud infrastructure and data engineering environments.

Key Technologies

Azure | Azure Databricks | Unity Catalog | Terraform | Python | Azure DevOps | GitHub Actions | Entra ID | Key Vault | Azure Networking | Azure Storage

Nice to Have

• Experience with Medallion Architecture (Bronze/Silver/Gold).
• Exposure to FinOps, Azure/Databricks cost optimisation or cost attribution.
• Experience with .NET is an advantage.
• Working knowledge of SQL.