Data Fabric engineer
Posted 1 week ago by LA International
£400 Per day
Inside
Remote
West Midlands
Summary: The Data Fabric Engineer role requires strong expertise in Microsoft Azure technologies, particularly Azure Fabric and Data Factory, along with proficiency in data transformation using PySpark, SQL, and Python. The position is fully remote and necessitates active SC clearance, with a focus on collaboration within agile teams and knowledge transfer to build client-side capabilities. Candidates with prior high-level security clearance are encouraged to apply, as security clearance is mandatory before appointment.
Key Responsibilities:
- Strong experience with Microsoft Azure Fabric (Lakehouses, Data Pipelines, Dataflows Gen2, Fabric Notebooks)
- Strong experience with Azure Data Factory (ADF) for orchestration and data movement
- Strong proficiency in PySpark, SQL, and Python for large-scale data transformation
- Strong experience with Delta Lake, Apache Spark, and OneLake architecture
- Good knowledge of Microsoft Purview for data governance, cataloguing, and lineage
- Good experience with Azure DevOps, Git-based version control, and CI/CD pipelines
- Good understanding of data lakehouse architecture (medallion architecture - Bronze/Silver/Gold)
- Experience with Azure Synapse Analytics or migration from Synapse to Fabric
- Familiarity with Databricks or equivalent distributed processing platforms
- Experience in UK public sector or government data environments
- Understanding of SC clearance requirements and government security classifications
- Knowledge of DDAT frameworks and GDS delivery standards
- Good knowledge of Azure storage services (ADLS Gen2, Azure Blob Storage)
- Ability to operate autonomously, prioritise competing workloads, and guide junior engineers
- Strong collaboration skills - able to work across multi-disciplinary agile teams
- Commitment to knowledge transfer and building client-side capability
- Familiarity with WCAG accessibility standards and government design principles
Key Skills:
- Microsoft Azure Fabric expertise
- Azure Data Factory proficiency
- Proficiency in PySpark, SQL, and Python
- Experience with Delta Lake and Apache Spark
- Knowledge of Microsoft Purview
- Experience with Azure DevOps and CI/CD pipelines
- Understanding of data lakehouse architecture
- Experience with Azure Synapse Analytics
- Familiarity with Databricks
- Experience in UK public sector data environments
- Understanding of SC clearance requirements
- Knowledge of DDAT frameworks and GDS standards
- Knowledge of Azure storage services
- Ability to guide junior engineers
- Strong collaboration skills
- Commitment to knowledge transfer
- Familiarity with WCAG standards
Salary (Rate): £400 per day
City: West Midlands
Country: United Kingdom
Working Arrangements: remote
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: IT
Here is the JD (SC Cleared only):
· Strong experience with Microsoft Azure Fabric (Lakehouses, Data Pipelines, Dataflows Gen2, Fabric Notebooks)
· Strong experience with Azure Data Factory (ADF) for orchestration and data movement
· Strong proficiency in PySpark, SQL, and Python for large-scale data transformation
· Strong experience with Delta Lake, Apache Spark, and OneLake architecture
· Good knowledge of Microsoft Purview for data governance, cataloguing, and lineage
· Good experience with Azure DevOps, Git-based version control, and CI/CD pipelines
· Good understanding of data lakehouse architecture (medallion architecture - Bronze/Silver/Gold)
· Experience with Azure Synapse Analytics or migration from Synapse to Fabric
· Familiarity with Databricks or equivalent distributed processing platforms
· Experience in UK public sector or government data environments
· Understanding of SC clearance requirements and government security classifications
· Knowledge of DDAT frameworks and GDS delivery standards
· Good knowledge of Azure storage services (ADLS Gen2, Azure Blob Storage)
· Ability to operate autonomously, prioritise competing workloads, and guide junior engineers
· Strong collaboration skills - able to work across multi-disciplinary agile teams
· Commitment to knowledge transfer and building client-side capability
· Familiarity with WCAG accessibility standards and government design principles
Qualifications:
· Microsoft Fabric Data Engineer Associate (DP-700)
Role is fully remote and requires active SC clearance
Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 10 weeks. LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.
An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.
Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.