Role / Responsibilities
- Define and build a data lake architecture on Azure (ADLS + Databricks)
- Layered data model (bronze / silver / gold) for ingestion, processing, and reporting
- Develop ingestion patterns from API sources (batch and streaming)
- Establish data governance, metadata, and access controls
- Define storage structures, naming conventions, and schema design
- Develop API connectors and ingestion pipelines using Python
- Implement integration services feeding into the Databricks environment
- Build data transformation logic using PySpark / Python (data cleansing, standardisation, validation)
- Configure monitoring, logging, and alerting
- Execute unit and integration testing across pipelines
Core Skills
- Azure
- Databricks
- Python
Desirable
- Any exposure to Financial Services, Insurance or Bank/Banking would be beneficial but not essential
Rate:
£500/day
Location:
Basingstoke
IR35 Status:
Outside
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Not Specified