Senior Data Engineer – Databricks & DLT

Senior Data Engineer – [Databricks, DLT (Delta Live Tables), Qlik Replicate]

Certain Advantage are recruiting on behalf of our trading client for a Senior Data Engineer with strong analytical capabilities and hands-on experience in building scalable data solutions using a programming language like Python or Scala.

We’re looking for candidates who can offer experience in the below so please ensure your application includes your experience with:

• Databricks, Delta Live Tables (DLT)
• ADF, and Qlik Replicate for data ingestion and replication.
• MS Copilot, AI Foundry , Databricks (MosiacAI, MLflow, Agentbricks, Genie).

Your Responsibilities

• Design and implement scalable data pipelines using Databricks and Delta Live Tables (DLT).
• Develop and manage streaming data solutions for real-time analytics and decision-making.
• Ingest data from various sources—including SAP systems using Qlik Replicate and ADF to ingest data from various sources ensuring data quality and consistency.
• Perform data modelling and optimize data structures for analytics and reporting.
• Collaborate with cross-functional teams to understand data requirements and deliver robust solutions.
• Monitor, troubleshoot, and enhance data workflows for performance and reliability.
• Maintain technical documentation and contribute to data engineering best practices.

Required Skills & Qualifications

• Strong experience with Databricks- DLT, Delta Lake concepts, UC governance.
• Solid understanding of streaming technologies (e.g., Spark Structured Streaming, Autoloader).
• Programming skills in Python, SQL
• Proficiency in data modelling, ETL/ELT processes, and data architecture.
• Experience in MS Copilot, AI Foundry , Databricks (MosiacAI, MLflow, Agentbricks, Genie)
• Strong analytical background with problem-solving skills.
• Performance tuning concepts like watermarking, late data handling, parallelism & checkpointing.
• Hands-on expertise in ADF, and Qlik Replicate for data ingestion and replication.
• Experience working in Azure cloud environments.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.