Data Engineer/Platform Engineer
We are seeking a skilled and experienced Senior Data/Platform Engineer to join our Data & Analytics team. This role combines hands-on data engineering on Databricks and Azure Synapse with platform administration responsibilities across our cloud data estate. The role holder will design, build, and operate scalable data pipelines while also maintaining the underlying Azure platform - including infrastructure-as-code (Pulumi), CI/CD automation, monitoring, security, and Databricks workspace administration. The ideal candidate combines strong Python/PySpark engineering skills with deep Azure platform knowledge and a service-excellence mindset, ensuring the Allianz Data Technology platform remains secure, reliable, and aligned to Allianz standards.
Working Mode: Remote with occasional office visit
Location: Guildford
Experience Required
- Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows).
- Python/PySpark: strong programming skills for data processing, automation, and Scripting.
- Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault - including Linked Services, Managed Identity, and Spark Pool configuration.
- Azure platform fundamentals: compute, storage, networking (VNets, NSGs, Private Endpoints), identity and RBAC.
- CI/CD: GitHub Actions (with OIDC federation) and/or Azure DevOps for data and platform deployments.
- Infrastructure-as-Code: Pulumi (or Terraform/Bicep) across multiple environments.
- Scripting: PowerShell and Bash for platform automation.
- Monitoring & observability: Azure Monitor, Log Analytics, KQL.
- Big data file formats: Parquet and Delta Lake.
- Cloud-native data modelling and ETL/ELT frameworks on Azure.