ETL SDET with Python and Pyspark
Posted Today by TekLeaders, Inc
Negotiable
Undetermined
Remote
Remote or Toronto, Ontario
Summary: The role of ETL SDET with Python and Pyspark involves ensuring the quality and reliability of ETL processes and data integration workflows. The candidate will design and implement automated tests, collaborate with data engineers and developers, and leverage expertise in ETL testing to enhance data pipeline performance. This position is fully remote and is expected to last for over 12 months. The ideal candidate will have significant experience in software development and testing within a cloud environment.
Key Responsibilities:
- Review requirements, specifications, and technical design documents to understand and provide feedback.
- Experience in developing and implementing test strategies for complex initiatives.
- Estimate, prioritise, plan and coordinate testing activities.
- Create automation test framework.
- Design, develop and execute automation scripts.
- Identify, record, document thoroughly and track bugs.
- Strong problem-solving skills, troubleshooting, and root cause analysis skills.
- Perform thorough regression testing when bugs are resolved.
- Develop and apply testing processes for new and existing features to meet client needs.
- Liaise with cross-functional teams to identify system requirements.
- Track quality assurance metrics, like defect densities and open defect counts.
- Stay up to date with new testing tools and test strategies requirements.
- Support and guide junior team members.
Key Skills:
- At least 8 years of IT experience with minimum 5 years of experience in SDET and ETL.
- Proficiency in ETL Testing and developing automation framework & test cases in at least one scripting language.
- Strong SQL skills, with experience in handling big data using ETL tools (e.g., Informatica).
- Testing experience in cloud environment (AWS S3).
- Experience with Test Automation and DevOps practices in ETL and Cloud.
- Work experience in a Big Data ecosystem processing data in data lakehouse.
- Familiarity with DBT (Data Build Tool) or similar data transformation tools is a plus.
- Experience providing testing oversight in Agile team.
- Excellent business communication skills to interact with Business teams and cross-functional teams.
Salary (Rate): undetermined
City: undetermined
Country: Canada
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Role: ETL SDET with Python and Pyspark
Remote 100% - Canada
12+ Months Long-term
Rate: on CAD
Skill Matrix
Skills
Years of Exp
Year lastly Used
Rating Out of 10
SDET (MUST)
ETL (MUST)
Python
Pyspark
AWS S3
Healthcare experience
Job Description
Position Summary:
The SDET (Software Development Engineer in Test) is responsible for ensuring the quality and reliability of ETL processes and data integration workflows.
This role involves designing and implementing automated tests to validate data transformation and migration, as well as collaborating with data engineers and developers to identify and resolve data quality issues.
The ideal candidate will leverage their expertise in ETL testing and data validation to enhance the overall data pipeline performance and integrity.
Minimum Qualification:
- At least 8 years of IT experience with minimum 5 years of experience in below skills:
- Tech Skills
- Proficiency in ETL Testing and developing automation framework & testcases in atleast one scripting language.
- Strong SQL skills, with experience in handling big data using ETL tools (e.g., Informatica).
- Testing experience in cloud environment (AWS s3)
- Experience with Test Automation and DevOps practices in ETL and Cloud
- Work experience in a Big Data ecosystem processing data in data lakehouse.
- Familiarity with DBT (Data Build Tool) or similar data transformation tools is a plus.
- Experience providing testing oversight in Agile team.
- Should have excellent business communication to interact with Business teams and cross-functional teams.
Responsibilities:
- Review requirements, specifications, and technical design documents to understand and provide feedback.
- Experience in developing and implementing test strategies for complex initiatives.
- Estimate, prioritise, plan and coordinate testing activities.
- Create automation test framework.
- Design, develop and execute automation scripts.
- Identify, record, document thoroughly and track bugs.
- Strong problem-solving skills, troubleshooting, and root cause analysis skills.
- Perform thorough regression testing when bugs are resolved.
- Develop and apply testing processes for new and existing features to meet client needs.
- Liaise with cross-functional teams to identify system requirements.
- Track quality assurance metrics, like defect densities and open defect counts.
- Stay up to date with new testing tools and test strategies requirements.
- Support and guide junior team members.