Test Lead - Cloud & Data Platform (Lakehouse)

Test Lead - Cloud & Data Platform (Lakehouse)

Posted 2 weeks ago by N Consulting Global

Apply
Negotiable
Undetermined
Hybrid
London Area, United Kingdom
Apply
Amazon Web Services (AWS) Automation AWS Glue AWS Identity And Access Management (IAM) AWS Lambda Azure Databricks Chaos Engineering Cloud Computing Cloud Technology Continuous Integration and Continuous Delivery Databricks Data Layers Data Pipeline Data Quality Extract Transform Load (ETL) Infrastructure as Code (IaC) Management Performance Testing Role-Based Access Control (RBAC) Snowflake (Data Warehouse) Stakeholder Management Test Automation Test Management Verification And Validation Workflows

Summary: The Test Lead / Test Manager will define and execute comprehensive testing strategies for a large-scale cloud data lakehouse on AWS, focusing on quality, reliability, security, and compliance. This role emphasizes a shift-left testing approach aligned with SRE principles and involves leading quality initiatives across various infrastructure and data layers. The position requires collaboration with cross-functional teams and a strong emphasis on test automation and CI/CD integration. The successful candidate will also provide leadership and mentoring throughout the testing lifecycle.

Key Responsibilities:

  • Lead the design and execution of end-to-end testing across a three-tier lakehouse architecture (landing, curated, analytics zones)
  • Define and implement DBT testing strategies, including schema validation, data assertions, and freshness checks
  • Oversee testing of AWS Glue ETL pipelines, ensuring transformation accuracy, incremental processing, and robust data quality controls
  • Validate Snowflake environments, including multi-tenant setups, warehouse performance, storage integrations, and RBAC models
  • Ensure reliability of event-driven orchestration workflows (EventBridge, Airflow/Astronomer, Step Functions), including exception and failure handling
  • Drive test automation, CI/CD integration, and quality engineering best practices
  • Collaborate with cross-functional teams including data engineers, platform engineers, and SRE teams
  • Provide leadership, mentoring, and governance across the testing lifecycle

Key Skills:

  • Strong experience in Test Leadership / Test Management roles
  • Hands-on experience with AWS data services and cloud-native testing approaches
  • Expertise in data pipeline testing, ETL validation, and data quality frameworks
  • Experience with DBT, Snowflake, and modern data lakehouse architectures
  • Familiarity with CI/CD pipelines, infrastructure-as-code testing, and automation frameworks
  • Strong understanding of event-driven architectures and orchestration tools
  • Excellent stakeholder management and communication skills

Salary (Rate): undetermined

City: London

Country: United Kingdom

Working Arrangements: hybrid

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Job Title: Test Lead / Test Manager

Work Model: Hybrid (2 days/week in office)

Location: London, UK

Engagement Type: Contract

Must have Skills: Cloud & Data Platform (Lakehouse)

Core Technical Skills

  • Terraform (IaC testing, unit tests, state management) | Advanced
  • GitLab CI/CD (pipeline testing, matrix jobs, artefacts) | Advanced
  • AWS data services (S3, IAM, Glue, Lake Formation, EventBridge, CloudWatch, Lambda) | Intermediate–Advanced
  • Security scanning tools (Checkov, tfsec) | Intermediate
  • Data platforms (Snowflake, Databricks, DBT, Apache Iceberg) | Intermediate
  • Python (test automation, pipeline utilities) | Intermediate
  • SRE practices (SLIs/SLOs, chaos engineering, performance testing) | Intermediate
  • Compliance testing (encryption, tagging, data residency) | Intermediate

Role Overview: As a Test Lead / Test Manager, you will be responsible for defining and executing end-to-end testing strategies to validate the quality, reliability, security, and compliance of a large-scale, multi-platform cloud data lakehouse on AWS. You will champion a shift-left testing approach aligned with SRE principles and lead quality initiatives across infrastructure, data, and platform layers.

Key Responsibilities:

  • Lead the design and execution of end-to-end testing across a three-tier lakehouse architecture (landing, curated, analytics zones)
  • Define and implement DBT testing strategies, including schema validation, data assertions, and freshness checks
  • Oversee testing of AWS Glue ETL pipelines, ensuring transformation accuracy, incremental processing, and robust data quality controls
  • Validate Snowflake environments, including multi-tenant setups, warehouse performance, storage integrations, and RBAC models
  • Ensure reliability of event-driven orchestration workflows (EventBridge, Airflow/Astronomer, Step Functions), including exception and failure handling
  • Drive test automation, CI/CD integration, and quality engineering best practices
  • Collaborate with cross-functional teams including data engineers, platform engineers, and SRE teams
  • Provide leadership, mentoring, and governance across the testing lifecycle

Key Skills & Experience:

  • Strong experience in Test Leadership / Test Management roles
  • Hands-on experience with AWS data services and cloud-native testing approaches
  • Expertise in data pipeline testing, ETL validation, and data quality frameworks
  • Experience with DBT, Snowflake, and modern data lakehouse architectures
  • Familiarity with CI/CD pipelines, infrastructure-as-code testing, and automation frameworks
  • Strong understanding of event-driven architectures and orchestration tools
  • Excellent stakeholder management and communication skills
Apply
Similar Jobs
Loading data...