Senior Data Engineer

Senior Data Engineer

Posted 1 day ago by Primus Connect Ltd

Apply
Negotiable
Outside
Remote
United Kingdom
Apply
Airflow Apache Airflow Apache Hadoop Apache Spark Azure Databricks Continuous Integration and Continuous Delivery Databricks Data Engineering Data Pipeline Github Microsoft Azure Python (Programming Language) SQL (Programming Language) Version Control Workflows

Summary: The role of Senior Data Engineer focuses on the Databricks Platform Buildout, requiring expertise in data engineering and consulting. The engineers will assess and prepare the Databricks platform for replication across various country-level platforms. This position is fully remote and classified as outside IR35.

Key Responsibilities:

  • Review group-level Databricks platform foundations for readiness.
  • Participate in the rollout and replication phase across country-level platforms.

Key Skills:

  • 4+ Years Data Engineering Experience.
  • 5+ years of consulting experience.
  • Minimum 2-3 projects delivered with hands-on experience in Databricks.
  • Experience in Spark and/or Hadoop.
  • Familiarity with Databricks multi-hop pipeline architecture.
  • Strong Databricks knowledge (PySpark, SQL, Delta Lake, Unity Catalog).
  • Proficiency in Python and SQL.
  • Experience with ETL/ELT development and data pipeline orchestration.
  • Knowledge of Github for version control.
  • Experience with CI/CD (Azure DevOps/Terraform/Bicep).

Salary (Rate): £575 daily

City: undetermined

Country: United Kingdom

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: Senior

Industry: IT

Detailed Description From Employer:

Databricks Platform Buildout

Senior Data Engineer

Outside IR35

Fully Remote

£500 - £575 per day

Overview & Objective

We are seeking 3x Senior Data Engineers for a Databricks Platform Buildout

project which is part of a bigger engagement.

The main project focus is to review the group-level Databricks platform foundations to confirm readiness for replication and then rollout across country-level platforms. These Engineers will participate in the rollout and replication phase.

Requirements

  • 4+ Years Data Engineering Experience,
  • 5+ years of consulting experience
  • Minimum 2-3 projects delivered with hands-on experience in Databricks
  • Experience in Spark and/or Hadoop,
  • Familiarity with Databricks multi-hop pipeline architecture

Technical Stack and Required Capabilities

  • Python, SQL
  • Strong Databricks knowledge (PySpark, SQL, Delta Lake, Unity Catalog)
  • Azure
  • ETL/ELT development and data pipeline orchestration (eg, Databricks Workflows, DLT, Airflow, ADF, Glue, and Step Functions.)
  • Github (creating branches, pull requests, reviewing PRs, solving merge conflicts)
  • Apache Spark
  • CI/CD (Azure DevOps/Terraform/Bicep
Apply
Similar Jobs
Loading data...