Data Quality Engineer/ETL Test Automation Lead - Inside IR35 | Databricks | Python | SQL
Posted 3 days ago by Hamilton Barnes
Negotiable
Inside
Hybrid
Stevenage, Hertfordshire, UK
Summary: The role of Data Quality Engineer/ETL Test Automation Lead involves leading the testing strategy for ETL/ELT pipelines within a major Life Sciences programme. This position focuses on building scalable automation frameworks and ensuring data quality across various layers of data processing. The candidate will work with modern technologies and have ownership of quality standards and engineering best practices. This is a greenfield opportunity with hands-on exposure to advanced tools and methodologies.
Key Responsibilities:
- Own and define the end-to-end testing strategy for ETL/ELT and data pipelines
- Design and implement scalable automation frameworks using Python and Pytest
- Build SQL-based validation and reconciliation checks
- Automate validation of Excel outputs using Pandas and OpenPyXL
- Validate XML outputs using lxml and xmlschema
- Develop and automate API testing using Postman and Newman
- Integrate testing into GitHub Actions and CI/CD pipelines
- Build reporting and logging capabilities for rapid defect diagnosis
- Collaborate closely with Data Engineers to improve quality and accelerate delivery
- Mentor team members and help establish QA standards and best practices
Key Skills:
- Strong experience testing ETL/ELT pipelines and complex data platforms
- Advanced SQL skills for validation, reconciliation, and troubleshooting
- Proven Python automation expertise using Pytest
- Experience with Pandas and OpenPyXL
- XML validation experience using lxml and xmlschema
- API testing experience with Postman and Newman
- Hands-on experience implementing CI/CD pipelines using GitHub Actions
- Experience with Databricks or similar modern data platforms
- Strong understanding of data quality, integrity, and transformation validation
- Experience building QA processes or leading data testing initiatives
Salary (Rate): undetermined
City: Stevenage
Country: UK
Working Arrangements: hybrid
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: Other
Location: Stevenage, UK (Hybrid - 3 days onsite)
Contract: 6 months
Rate: Competitive Day Rate - Negotiable
Start: ASAP
Industry: Life Sciences
We are seeking an experienced Data Quality Engineer/ETL Test Automation Lead to join a major Life Sciences programme delivering business-critical data solutions. This is a greenfield opportunity where you will establish and lead the testing strategy for modern ETL/ELT pipelines and build scalable automation frameworks from the ground up.
You will work across the full data life cycle, ensuring quality and reliability across ingestion, transformation, and delivery layers. This includes validating APIs, Excel outputs, XML files, and large-scale data processing environments.
This role offers hands-on exposure to modern technologies including Databricks, Python, Pytest, SQL, Pandas, GitHub Actions, and CI/CD automation while giving you ownership of quality standards and engineering best practices.
Key Responsibilities:
- Own and define the end-to-end testing strategy for ETL/ELT and data pipelines
- Design and implement scalable automation frameworks using Python and Pytest
- Build SQL-based validation and reconciliation checks
- Automate validation of Excel outputs using Pandas and OpenPyXL
- Validate XML outputs using lxml and xmlschema
- Develop and automate API testing using Postman and Newman
- Integrate testing into GitHub Actions and CI/CD pipelines
- Build reporting and logging capabilities for rapid defect diagnosis
- Collaborate closely with Data Engineers to improve quality and accelerate delivery
- Mentor team members and help establish QA standards and best practices
Required Skills & Experience:
- Strong experience testing ETL/ELT pipelines and complex data platforms
- Advanced SQL skills for validation, reconciliation, and troubleshooting
- Proven Python automation expertise using Pytest
- Experience with Pandas and OpenPyXL
- XML validation experience using lxml and xmlschema
- API testing experience with Postman and Newman
- Hands-on experience implementing CI/CD pipelines using GitHub Actions
- Experience with Databricks or similar modern data platforms
- Strong understanding of data quality, integrity, and transformation validation
- Experience building QA processes or leading data testing initiatives
Desirable:
- Exposure to cloud-based data environments
- Experience with Great Expectations or similar data quality tools
- Knowledge of Airflow or Azure Data Factory
- Performance and high-volume data testing
- End-to-end validation experience across APIs, files, and pipelines