Summary: The role of Azure Data Engineer requires candidates to possess SC clearance and involves working with Azure data services, Databricks, and SQL for data ingestion, transformation, and reporting. The position emphasizes collaboration with analysts and data scientists to refine metrics and develop data models. Responsibilities include creating ADF pipelines and stored procedures, as well as ensuring adherence to data standards for ETL processes. This is a contract position lasting over 3 months, with a focus on delivering high-quality data solutions.

Key Responsibilities:

• Analyse raw data for parsing, schema evolution, and transformation towards metric development.
• Assess reporting/metric requirements from a data engineering perspective for refinement and deployment.
• Collaborate with analysts and data scientists to understand business requirements and data sources.
• Create normalized/dimensional data models based on requirements.
• Refine notebooks and logic developed as part of prototypes.
• Transform data from various stages to synapse dimensional models.
• Create notebooks in Databricks for incremental data load and transformation.
• Develop stored procedures for data load and transformation in Azure Synapse dedicated pools.
• Create ADF pipelines for data orchestration across Databricks and Synapse.

Key Skills:

• Strong experience with Azure data services (ADF, Synapse, SQL, ADB).
• Proficient in Databricks notebook development for data ingestion and transformation.
• Expertise in PySpark and SQL.
• Experience in ADF pipeline development and data orchestration techniques.
• Strong skills in stored procedure development.
• Good knowledge of dimensional data modeling and Power BI reporting.

Salary (Rate): 450

City: undetermined

Country: UK

Working Arrangements: remote

IR35 Status: inside IR35

Seniority Level: undetermined

Industry: IT