Databricks Consultant

Summary: The role of Databricks Consultant involves leading the design, implementation, and optimization of a modern cloud-based data platform. This hands-on contractor position requires proven experience in delivering enterprise Databricks solutions from design to production. The position is fully remote and classified as outside IR35, with an initial contract duration of six months.

Key Responsibilities:

• Design and implement scalable Databricks Lakehouse solutions.
• Build and optimise ETL/ELT pipelines using PySpark and SQL.
• Implement Delta Lake, Unity Catalog and Medallion Architecture.
• Develop batch and streaming data ingestion pipelines.
• Configure CI/CD, Infrastructure as Code and automated deployments.
• Optimise Spark performance, security and cost.
• Implement data governance, monitoring and best practices.
• Produce technical documentation and mentor internal teams.

Key Skills:

• Proven experience delivering enterprise Databricks implementations.
• Strong Databricks expertise including Delta Lake, Unity Catalog, Workflows and Delta Live Tables.
• Expert knowledge of PySpark, Python and SQL.
• Strong Apache Spark performance tuning experience.
• Experience designing modern cloud data platforms and Lakehouse architectures.
• Experience building batch and real-time data pipelines.
• Strong understanding of data modelling, ETL/ELT and data warehousing concepts.
• Experience implementing CI/CD using Azure DevOps, GitHub or GitLab.
• Infrastructure as Code using Terraform.
• Strong knowledge of data governance, security and access control.

Salary (Rate): undetermined

City: undetermined

Country: United Kingdom

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT