Role Summary
We are seeking an experienced Databricks Lead Engineer to drive the design, development, and optimization of enterprise-scale data platforms.
The ideal candidate will have strong technical leadership capabilities, deep expertise in the Databricks Lakehouse platform, and a proven track record of delivering robust data pipelines and architecture in complex environments.
Key Responsibilities
- Lead the design and implementation of scalable data architectures on the Databricks Lakehouse platform.
- Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake.
- Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning.
- Collaborate with data architects, analysts, and stakeholders to translate business requirements into technical data solutions.
- Ensure data security, governance, and quality standards are maintained across all data workflows.
- Attend the London office for 3 days per week for collaborative working and stakeholder engagement.
Essential Requirements & Skills
- Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization.
- Programming Languages: Advanced proficiency in Python/PySpark and SQL.
- Cloud & Infrastructure: Strong background in cloud data platforms (Azure, AWS, or GCP) and cloud-native architectures.
- Engineering Best Practices: Experience with CI/CD pipelines, version control (Git), and infrastructure-as-code.
- Leadership: Demonstrated experience leading engineering teams, managing workstreams, and driving technical delivery.
Contract & Commercial Terms
- Rate: Up to £575 per day.
- Tax Status: Inside IR35.
- Duration: 6 months (with scope for extension).
Rate:
£575/day
Location:
London
IR35 Status:
Inside
Remote Status:
Hybrid
Industry:
Data & Analytics
Seniority Level:
Senior