Azure Databricks Data Engineer - OUTSIDE IR35

Summary: The role of a Databricks Data Engineer involves working on data engineering tasks within the Databricks environment, specifically for capital markets or banking sectors. The position requires expertise in Spark programming and familiarity with Azure Data Lake, along with a strong emphasis on data governance and architecture. An immediate start is expected, with a hybrid working arrangement based in London.

Key Responsibilities:

• Data Engineering in Databricks, including Spark programming with Scala, Python, and SQL.
• Experience with Delta Lake and Databricks workflows.
• Data ingestion and ETL/ELT frameworks using Azure Data Lake.
• Implementing data governance practices, including metadata management and data quality assurance.
• Understanding data modeling concepts and managing data products and domains.
• Utilizing Unity Catalog or similar data governance management components.
• Experience with MS Purview or similar tools for metadata and data quality.
• Driving implementation and solution design for data architecture.

Key Skills:

• Proficiency in Databricks, Spark, Scala, Python, and SQL.
• Experience with Delta Lake and Azure Data Lake.
• Knowledge of data governance, metadata, and data quality tools.
• Understanding of data modeling and data architecture.
• Ability to drive implementation and solution design.
• Experience with Unity Catalog or similar data governance components.
• Familiarity with MS Purview and other data quality tools.

Salary (Rate): £550

City: London

Country: UK

Working Arrangements: hybrid

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT