Role / Responsibilities
Define and build a data lake architecture on Azure (ADLS + Databricks)
Layered data model (bronze / silver / gold) for ingestion, processing, and reporting
Develop ingestion patterns from API sources (batch and streaming)
Establish data governance, metadata, and access controls
Define storage structures, naming conventions, and schema design
Develop API connectors and ingestion pipelines using Python
Implement integration services feeding into the Databricks environment
Build data transformation logic using PySpark / Python (data cleansing, standardisation, validation)
Configure monitoring, logging, and alerting
Execute unit and integration testing across pipelines
Core Skills
- Azure
- Databricks
- Python
Desirable
Any exposure to Financial Services, Insurance or Bank/Banking would be beneficial but not essential