Data Engineer
Posted 3 days ago by Hancock & Parsons Ltd
£81 Per hour
Outside
Onsite
Edinburgh, Scotland, United Kingdom
Summary: The role involves two experienced Data Engineers for a well-established IT consultancy and Microsoft partner, focusing on data ingestion and processing. The positions are for an initial 6-month period and require on-site work in Edinburgh for four days a week, with one day working from home. Candidates should possess core skills in Databricks, SQL, and CI/CD DevOps pipelines, among others. The roles are classified as outside IR35.
Key Responsibilities:
- Utilize Databricks/Spark for real-time data ingestion and processing.
- Optimize and transform SQL queries.
- Implement CI/CD DevOps pipelines.
- Work with Event Hubs for streaming ingestion.
- Utilize Python in the Databricks context.
- Orchestrate data workflows using Azure Data Factory (ADF).
- Integrate governance solutions with Collibra (optional).
Key Skills:
- Experience with Databricks/Spark.
- Proficiency in SQL.
- Knowledge of CI/CD DevOps pipelines.
- Experience with Event Hubs.
- Familiarity with Python in a Databricks context.
- Experience with Azure Data Factory (ADF).
- Understanding of Collibra for governance integration (optional).
Salary (Rate): £81.00/hr
City: Edinburgh
Country: United Kingdom
Working Arrangements: on-site
IR35 Status: outside IR35
Seniority Level: Mid-Level
Industry: IT
A well established IT consultancy and MS partner are seeking two experienced Data Engineers for an initial period of 6 months outside IR35. Both roles are based in Edinburgh as there is a requirement to work on-site 4 days per week (1 day from home).
Core Skills:
- Databricks / Spark (preferred for real-time data ingestion and processing)
- SQL (optimisation and transformation)
- CI/CD DevOps pipelines
- Event Hubs (preferred for streaming ingestion)
Preferred:
- Python (Databricks context)
- Azure Data Factory (ADF) for orchestration
- Collibra (optional, governance integration)