AWS Glue ETL Developer
Posted 1 day ago by 1763534640
Negotiable
Outside
Remote
USA
Summary: The Sr. AWS Glue ETL Developer role focuses on re-engineering mainframe batch jobs into ETL processes using AWS Glue and other cloud-native tools. The position requires collaboration with mainframe resources, designing Glue jobs, and implementing solutions that include PySpark jobs and various AWS services. Strong communication skills and a solid background in ETL development and AWS are essential for success in this fully remote position. Candidates should have significant experience in software development and a relevant degree.
Key Responsibilities:
- Re-engineer mainframe batch jobs to equivalent ETL batch processes, preferably using Cloud-Native tools like AWS Glue.
- Leverage serverless cloud services to prepare (extract and transform) and load large numbers of datasets for data processing.
- Coordinate closely with mainframe resources to ensure requirements and specifications are clearly understood.
- Design Glue jobs to meet requirements, consider performance where applicable, code per standards, and perform end-to-end testing.
- Document the design specifications and contribute to implementation plan then work with the architecture team and lead developer to review the design and the code.
- Effectively coordinate the infrastructure/service needs working with the architecture and infrastructure teams.
- Implement overall solution comprising of Glue ETL PySpark jobs including SQL queries, Lambda, Step Functions, REST API calls.
- Develop and integrate custom developed software solutions to leverage automated deployment technologies.
- Use industry leading DevOps tools such as CodeCommit, CodeBuild, CodePipeline and AWS CloudFormation.
- Communicate key project data to team members and infrastructure team cohesion and effectiveness.
- Leverage Atlassian tool suite like JIRA and Confluence to track activities.
- Identify and apply best practices and standard operating procedures.
- Create innovative solutions to meet the technical needs of customers.
Key Skills:
- Experience working in the software development lifecycle, with strong experience on ETL based development.
- Working experience with development using Amazon Web Services (AWS) and primarily on AWS Glue studio.
- Experience working with programming language PySpark/Python.
- Experience working with relational database like PostgreSQL.
- Experience working in a fast-paced development environment with drive to completion.
- Well versed with using version control systems (CodeCommit, CodeBuild, CodePipeline preferred).
- Well versed with using issue/problem tracking systems (Jira preferred).
- Candidate must have bachelor’s with 8-12 years of prior relevant experience or master’s with 6-10 years of prior relevant experience.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Detailed Description From Employer:
Sr. AWS Glue ETL Developer
Job Description:
- Re-engineer mainframe batch jobs to equivalent ETL batch processes, preferably using Cloud-Native tools like AWS Glue.
- Leverage serverless cloud services to prepare (extract and transform) and load large numbers of datasets for data processing.
- Coordinate closely with mainframe resources to ensure requirements and specifications are clearly understood.
- Design Glue jobs to meet requirements, consider performance where applicable, code per standards, and perform end-to-end testing.
- Document the design specifications and contribute to implementation plan then work with the architecture team and lead developer to review the design and the code.
- Effectively coordinate the infrastructure/service needs working with the architecture and infrastructure teams.
- Implement overall solution comprising of Glue ETL PySpark jobs including SQL queries, Lambda, Step Functions, REST API calls.
- Develop and integrate custom developed software solutions to leverage automated deployment technologies.
- Use industry leading DevOps tools such as CodeCommit, CodeBuild, CodePipeline and AWS CloudFormation.
- Communicate key project data to team members and infrastructure team cohesion and effectiveness.
- Leverage Atlassian tool suite like JIRA and Confluence to track activities.
- Identify and apply best practices and standard operating procedures.
- Create innovative solutions to meet the technical needs of customers.
Basic Qualifications:
- Experience working in the software development lifecycle, with strong experience on ETL based development.
- Working experience with development using Amazon Web Services (AWS) and primarily on AWS Glue studio.
- Experience working with programming language PySpark/Python
- Experience working with relational database like PostgreSQL.
- Experience working in a fast-paced development environment with drive to completion.
- Well versed with using version control systems (CodeCommit, CodeBuild, CodePipeline preferred)
- Well versed with using issue/problem tracking systems (Jira preferred)
- Candidate must have bachelor s with 8-12 years of prior relevant experience or master s with 6-10 years of prior relevant experience
Preferred Qualifications:
- Familiarity working with multiple file formats, and knowledge on DB2, JCL, COBOL COPYBOOK and EBCDIC character format is strongly desired.
- AWS, Agile, or other related Certification(s)
- Prior experience working mainframe modernization.
- Prior experience with Linux bash scripting
Manager Notes:
100% Remote role
Need strong candidates with good communication skills
Needs to have all the skills and tools listed
Bachelors degree is required
Minimum 8+ years of exp.