Technical Lead - Data Engineering

The Role

Technical Lead - Data Engineering and play a critical role in driving our enterprise data platform strategy.

You will lead the design, development, and delivery of modern data solutions using Snowflake, DBT, Azure/AWS, Python, Airflow, and CI/CD technologies.

This role combines deep technical expertise with leadership responsibilities, guiding engineering teams, defining best practices, and ensuring the successful delivery of scalable, secure, and high-performing data platforms.

Your responsibilities:

• Lead the design and implementation of scalable and secure data platforms using Snowflake, DBT, Airflow, Python, and Azure/AWS services.
• Define technical architecture, coding standards, engineering best practices, and development frameworks for the data engineering team.
• Drive the adoption of CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, or similar tools to automate build, test, deployment, and release management processes.
• Lead the development, optimization, and maintenance of complex ETL/ELT pipelines for large-scale data processing.
• Establish DevOps and DataOps practices to improve deployment efficiency, reliability, and operational excellence.
• Mentor and coach data engineers through technical guidance, code reviews, architecture reviews, and knowledge-sharing sessions.
• Collaborate with enterprise architects and business stakeholders to translate business requirements into scalable technical solutions.
• Own platform reliability, monitoring, performance tuning, and troubleshooting of production data pipelines.
• Implement Infrastructure as Code (IaC) using Terraform/Terragrunt to automate cloud resource provisioning.
• Drive data quality, governance, security, and compliance standards across the data ecosystem.
• Lead technical discussions, solution design workshops, and project planning activities.
• Evaluate emerging technologies and recommend innovative approaches to improve data engineering capabilities and delivery processes.
• Support Agile delivery and provide technical leadership throughout the project life cycle.

Technical Leadership

• Proven experience as a Technical Lead, Lead Data Engineer, or similar leadership role.
• Experience leading distributed development teams and delivering large-scale data engineering projects.
• Strong stakeholder management and technical decision-making capabilities.

Python & Data Engineering

• Expert-level proficiency in Python for data engineering, automation, orchestration, and application development.
• Strong experience developing scalable ETL/ELT frameworks using Python and SQL.
• Hands-on experience with DBT, Airflow, Snowflake, and cloud-native data services.

CI/CD & DevOps

• Strong experience designing and implementing CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, GitLab CI/CD, or similar platforms.
• Experience implementing automated testing, code quality checks, release management, and deployment automation.
• Strong understanding of DevOps, DataOps, CI/CD best practices, and release governance.

Cloud & Platform Engineering

• Extensive experience designing cloud-based data solutions on Azure and/or AWS.
• Strong knowledge of cloud security, networking, monitoring, and operational best practices.
• Experience with Infrastructure as Code using Terraform and Terragrunt.

Data Architecture

• Expertise in Data Vault, dimensional modelling, data warehousing, and modern data platform architectures.
• Advanced SQL development and performance optimization skills.
• Experience building enterprise-grade data products and analytics platforms.

Version Control & Engineering Practices

• Strong Git/GitHub experience, including branching strategies, pull requests, code reviews, and release processes.
• Experience implementing engineering standards, quality gates, and development best practices.

Communication & Stakeholder Engagement

• Excellent communication and presentation skills.
• Ability to engage with business and technical stakeholders at all levels.
• Strong problem-solving, analytical thinking, and decision-making capabilities.

Desirable Skills/Knowledge/Experience

• Experience with Generative AI and AI-powered data engineering solutions.
• Experience with Power BI, MicroStrategy, or other BI tools.
• Knowledge of Kubernetes, Docker, and containerized deployments.
• Experience with Databricks and modern lakehouse architectures.
• Azure Data Factory, Synapse Analytics, or AWS Glue experience.
• Experience implementing DataOps frameworks and observability platforms.
• Exposure to enterprise architecture and governance frameworks.
• Languages: Python (primary), SQL, Bash
• Cloud: Azure, AWS
• Tools: Airflow, DBT
• Data: Snowflake, Delta Lake, Redis, Azure Data Lake
• Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor.