ETL SDET - PySpark Coding Experience
Posted 1 day ago by 1761905209
Negotiable
Outside
Remote
USA
Summary: The ETL SDET role focuses on ensuring the quality and reliability of ETL processes and data integration workflows. The position involves designing and implementing automated tests, collaborating with data engineers, and enhancing data pipeline performance. Candidates should have extensive experience in ETL testing, Python, and SQL. This is a contract role with remote work options available.
Key Responsibilities:
- Design and implement automated tests for ETL processes.
- Collaborate with data engineers and developers to resolve data quality issues.
- Enhance overall data pipeline performance and integrity.
- Provide testing oversight in Agile teams.
Key Skills:
- ETL Testing and automation framework development.
- Proficiency in Python and PySpark.
- Strong SQL skills and experience with big data ETL tools.
- Experience in cloud environments, particularly AWS S3.
- Familiarity with DBT or similar data transformation tools.
- Excellent business communication skills.
Salary (Rate): undetermined
City: New York
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Hello Everyone,
Hope you are doing good
This is Raja from Tek Leaders. We have an urgent requirement below that is open with our client.
Role: ETL SDET
Location: Remote.
Contract role
Must-Have Skills:
- ETL Testing, PythonPySpark - Coding Experience is Mandatory
- AWS S3, Redshift
- Complex SQL Quires - Wring Complex Query is Mandatory
Good to Have:
- Kafka, Shell scripting, Healthcare domain
Position Summary:
The SDET (Software Development Engineer in Test) is responsible for ensuring the quality and reliability of ETL processes and data integration workflows.
This role involves designing and implementing automated tests to validate data transformation and migration,
as well as collaborating with data engineers and developers to identify and resolve data quality issues.
The ideal candidate will leverage their expertise in ETL testing and data validation to enhance the overall data pipeline performance and integrity.
Minimum Qualification:
- At least 12 years of IT experience with minimum 8 years of experience in below skills:
- Tech Skills
- Proficiency in ETL Testing and developing automation framework & testcases in atleast one scripting language.
- Strong SQL skills, with experience in handling big data using ETL tools (e.g., Informatica).
- Testing experience in cloud environment (AWS s3)
- Experience with Test Automation and DevOps practices in ETL and Cloud
- Work experience in a Big Data ecosystem processing data in data lakehouse.
- Familiarity with DBT (Data Build Tool) or similar data transformation tools is a plus.
- Experience providing testing oversight in Agile team.
- Should have excellent business communication to interact with Business teams and cross-functional teams.
Skill Matrix
|
Skills |
Years of Exp |
Year lastly Used |
Rating Out of 10 |
|
SDET (MUST) |
|
|
|
|
ETL (MUST) |
|
|
|
|
Python |
|
|
|
|
Pyspark |
|
|
|
|
AWS S3 |
|
|
|
|
Strong SQL |
|
|
|