Senior Azure Data Engineer

Job Description

We are seeking a Senior Data Engineer to support the design, development, and maintenance of enterprise data platforms within a Microsoft Azure cloud environment. The ideal candidate will be responsible for designing scalable data architectures, developing ELT/ETL pipelines, optimizing cloud-based data solutions, and enabling advanced analytics and machine learning initiatives.

The candidate will work closely with Data Scientists and business stakeholders to build secure, reliable, and high-performing data pipelines while following modern data engineering best practices.

Responsibilities

• Developing reusable code
• Implementing source control and CI/CD workflows
• Ensuring data quality
• Creating documentation
• Supporting analytics through Azure Synapse Analytics, Azure Machine Learning, and Azure Data Lake Storage
• Optimizing data processing
• Implementing monitoring and error handling
• Maintaining data architecture
• Evaluating emerging AI and automation technologies to improve data engineering processes

Required Skills

• 5+ years of experience designing, implementing, and maintaining ELT/ETL pipelines
• Strong experience with Microsoft Azure cloud services
• Hands-on experience with Azure Synapse Analytics
• Experience with Azure Machine Learning (SDK v1 & v2)
• Strong SQL and T-SQL programming skills
• Proficiency in Python (Pandas required; PySpark or Polars preferred)
• Experience developing reusable, modular code
• Experience designing and maintaining cloud-based data architectures
• Expertise in Azure Data Lake Storage (ADLS)
• Data modeling and database optimization
• Pipeline monitoring, logging, validation, and error handling
• Source control using Git
• CI/CD pipeline implementation
• Code-first development using Python SDK, CLI, REST APIs, or Infrastructure as Code (IaC)
• Experience with machine learning data pipelines
• Knowledge of Parquet and modern data formats
• Documentation including data dictionaries, ER diagrams, SOPs, and process documentation
• Experience collaborating with Data Scientists and business stakeholders
• Familiarity with AI coding assistants and LLM integration patterns (preferred)

Preferred Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Data Science, Machine Learning, Mathematics, or a related field (or equivalent experience)
• Azure certifications such as DP-203 or equivalent
• Experience implementing CI/CD and DevOps practices
• Experience with Infrastructure as Code (IaC)
• Experience integrating AI/LLM capabilities into data engineering solutions