Data Engineer (MS Fabric experience essential)
Posted Today by Jobserve
£400 Per day
Inside
Remote
England
Summary: The role of Data Engineer requires strong expertise in Microsoft Azure Fabric, focusing on data engineering tasks predominantly in a remote setting with occasional client visits. Candidates must possess active SC clearance and demonstrate proficiency in various data technologies, including Azure Data Factory and PySpark. The position is classified as inside IR35, indicating specific tax implications for contractors. The employer emphasizes inclusivity and encourages applications from diverse backgrounds.
Key Responsibilities:
- Utilize Microsoft Azure Fabric for data engineering tasks including Lakehouses, Data Pipelines, and Dataflows Gen2.
- Orchestrate and manage data movement using Azure Data Factory (ADF).
- Transform large-scale data using PySpark, SQL, and Python.
- Implement Delta Lake, Apache Spark, and OneLake architecture.
- Govern data using Microsoft Purview for cataloguing and lineage.
- Manage version control and CI/CD pipelines with Azure DevOps and Git.
- Understand and apply data lakehouse architecture principles.
- Work with Azure Synapse Analytics and facilitate migration to Fabric.
- Utilize Databricks or similar distributed processing platforms.
Key Skills:
- Strong experience with Microsoft Azure Fabric.
- Proficiency in Azure Data Factory (ADF).
- Expertise in PySpark, SQL, and Python.
- Experience with Delta Lake and Apache Spark.
- Knowledge of Microsoft Purview.
- Familiarity with Azure DevOps and CI/CD pipelines.
- Understanding of data lakehouse architecture.
- Experience with Azure Synapse Analytics.
- Familiarity with Databricks or equivalent platforms.
Salary (Rate): £400 per day
City: undetermined
Country: England
Working Arrangements: remote
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: IT
We have a new requirement for a Data Engineer with strong experience usuing Microsoft Azure Fabric.
This role has been deemed as Inside IR35 and will be predominantly remote working with occasional travel to client locations
Active SC clearance is essential
Skills:
Strong experience with Microsoft Azure Fabric (Lakehouses, Data Pipelines, Dataflows Gen2, Fabric Notebooks)
Strong experience with Azure Data Factory (ADF) for orchestration and data movement
Strong proficiency in PySpark, SQL, and Python for large-scale data transformation
Strong experience with Delta Lake, Apache Spark, and OneLake architecture
Good knowledge of Microsoft Purview for data governance, cataloguing, and lineage
Good experience with Azure DevOps, Git-based version control, and CI/CD pipelines
Good understanding of data lakehouse architecture (medallion architecture - Bronze/Silver/Gold)
Experience with Azure Synapse Analytics or migration from Synapse to Fabric
Familiarity with Databricks or equivalent distributed processing platforms
To be considered please submit an up-to-date CV
Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 10 weeks. LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.
An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.
Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.