Data Engineer (ETL, Python, AWS)

Data Engineer (ETL, Python, AWS)

Posted 1 day ago by 1762410079

Apply
Negotiable
Outside
Remote
USA
Apply
Amazon DynamoDB Amazon Web Services (AWS) Apache Hadoop Apache Kafka Apache Spark AWS Certified Solutions Architect AWS Developer AWS Elastic MapReduce (EMR) AWS Glue AWS Lambda AWS Solution Architect Big Data Cloud Computing Cloud Technology Continuous Integration and Continuous Delivery Data Manipulation Extract Transform Load (ETL) On Prem Pyspark Python (Programming Language) Software Coding Software Development SQL (Programming Language) Workflows

Summary: The Data Engineer role focuses on designing, developing, and implementing ETL solutions using Python and PySpark within AWS data services. The position requires strong communication skills and collaboration, as well as hands-on experience with AWS tools and SQL for data manipulation. Candidates should have experience leading teams and familiarity with big data technologies. This is a remote, long-term contract position outside IR35.

Key Responsibilities:

  • Design, develop, and implement robust ETL solutions using Python and PySpark.
  • Extract, transform, and load data from various sources into AWS data services.
  • Utilize SQL for data manipulation, querying, and optimization across various database platforms.
  • Hands-on experience with AWS Glue, EMR, Step Functions, and Lambda for ETL workflows.
  • Implement CI/CD pipelines using AWS CDK or similar tools.
  • Lead or mentor a team of developers/engineers in a collaborative environment.
  • Troubleshoot complex technical issues and provide operational support in AWS.

Key Skills:

  • Proficiency in Python and PySpark.
  • Experience with AWS Glue, EMR, Step Functions, and Lambda.
  • Strong SQL skills for data manipulation and querying.
  • AWS certifications such as AWS Certified Developer or AWS Certified Solutions Architect.
  • Familiarity with big data technologies like Apache Spark, Hadoop, or Kafka.
  • Excellent communication and collaboration skills.

Salary (Rate): undetermined

City: undetermined

Country: USA

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:
Position: Data engineer
Remote role
Long term Contract
Interview: 2 step process and will require live coding exercise
Note: Python/PySpark, AWS Glue and SQL
Details
Candidate must have great communication skills and be highly collaborative.
Design, develop, and implement robust ETL solutions using Python and PySpark to extract, transform, and load data from various sources into AWS data services. Expertise in SQL for data manipulation, querying, and optimization to work with various database platforms including Postgres, DynamoDB, Oracle, and Redshift.
Hands-on experience with AWS Glue, EMR, Step Functions, and Lambda for building and orchestrating ETL workflows in a cloud environment.
Implementing CI/CD pipelines using AWS CDK or similar tools for automating deployment and testing ETL solutions.
Certification(s): AWS certifications such as AWS Certified Developer or AWS Certified Solutions Architect
Previous experience leading or mentoring a team of developers/engineers in a collaborative environment.
Familiarity with big data technologies such as Apache Spark, Hadoop, or Kafka for processing large-scale datasets.
Troubleshoot complex technical issues and provide advanced operational support to internal in AWS
They are going from on prem to the cloud so the AWS, ETL, SQL and Python is a must.
Apply
Similar Jobs
Loading data...