Lead Data Engineer
Posted Today by Uniting Ambition
Negotiable
Outside
Onsite
Birmingham, England, United Kingdom
Summary: The Lead Data Engineer will oversee the migration of customer data from a third-party platform to a new Databricks environment, requiring both leadership and hands-on technical expertise. This role demands a strong background in Databricks and Azure data services, along with the ability to define data architecture strategies and ensure best practices. The position is based in Birmingham and offers a contract outside IR35 for an initial duration of six months. An ASAP start is preferred for the right candidate.
Key Responsibilities:
- Lead the migration of customer data from an external third-party provider to the new Databricks platform.
- Define and implement the end-to-end data architecture strategy within Databricks.
- Design scalable, secure, and high-performance data pipelines.
- Establish governance, data modelling standards, and best practice frameworks.
- Act as technical authority and SME for Databricks across the programme.
- Provide architectural oversight while remaining hands-on with engineering delivery.
Key Skills:
- Deep hands-on expertise in Databricks (architecture, optimisation, deployment).
- Deep expertise in Azure data services (Databricks, Data Factory, ADLS) and strong understanding of distributed data processing.
- Hands-on experience with CI/CD tooling (Azure DevOps/GitHub) and modern DevOps practices for data engineering.
- Strong grasp of data architecture, ETL/ELT design, and performance optimisation.
- Proven experience leading enterprise data platform implementations.
- Strong experience migrating customer data between platforms.
- Experience operating in senior/lead/principal-level engineering roles.
- Strong understanding of data governance, security, and compliance frameworks.
- Comfortable leading small, high-performing teams.
- Experience working within retail, eCommerce, or customer-focused environments (preferred).
Salary (Rate): undetermined
City: Birmingham
Country: United Kingdom
Working Arrangements: on-site
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Lead Data Engineer Contract Outside IR35 6 Months Initially 2/3 days a week on-site - Birmingham ASAP Start
My client are seeking an experienced Lead Data Engineer to lead the migration of customer data from a third-party platform onto the new Databricks environment. This is a hands-on leadership role requiring deep Databricks expertise combined with architectural ownership and delivery accountability.
Key Responsibilities
- Lead the migration of customer data from an external third-party provider to the new Databricks platform.
- Define and implement the end-to-end data architecture strategy within Databricks.
- Design scalable, secure, and high-performance data pipelines.
- Establish governance, data modelling standards, and best practice frameworks.
- Act as technical authority and SME for Databricks across the programme.
- Provide architectural oversight while remaining hands-on with engineering delivery.
Required Experience
- Deep hands-on expertise in Databricks (architecture, optimisation, deployment).
- Deep expertise in Azure data services (Databricks, Data Factory, ADLS) and strong understanding of distributed data processing.
- Hands-on experience with CI/CD tooling (Azure DevOps/GitHub) and modern DevOps practices for data engineering.
- Strong grasp of data architecture, ETL/ELT design, and performance optimisation.
Desirable Experience
- Proven experience leading enterprise data platform implementations.
- Strong experience migrating customer data between platforms.
- Experience operating in senior/lead/senior/lead/principle-level engineering roles.
- Strong understanding of data governance, security, and compliance frameworks.
- Comfortable leading small, high-performing teams.
- Experience working within retail, eCommerce, or customer-focused environments (preferred).