ETL developer with AWS Cloud

ETL developer with AWS Cloud

Posted 4 days ago by 1754039877

Apply
Negotiable
Outside
Remote
USA
Apply
Access Controls Airflow Amazon Managed Streaming for Apache Kafka (Amazon MSK) Amazon Relational Database Services Amazon S3 Amazon Web Services (AWS) Apache Airflow Apache Cassandra Apache Spark AWS CLI (Command Line Interface) AWS Elastic MapReduce (EMR) AWS Identity And Access Management (IAM) AWS SageMaker Big Data Cloud Computing Cloud Technology Data Lakes Data Modeling Data Pipeline Data Processing Data Quality Data Store Erwin (Data Modeling Software) Extract Transform Load (ETL) File Servers IBM DB2 Microsoft SQL Servers Scalability Scheduling Snowflake (Data Warehouse) Software Development SQL (Programming Language) Sqoop Systems Architecture Talend Verification And Validation Workflows

Summary: The role of an ETL Developer with AWS Cloud focuses on designing, building, and automating ETL processes using various AWS services and tools. The position requires extensive experience in data pipeline development, ensuring data quality, and optimizing workflows for performance and cost efficiency. The developer will also be responsible for implementing security measures and documenting processes. This role is fully remote and requires strong SQL skills along with proficiency in big data tools and AWS services.

Key Responsibilities:

  • Designing, building, and automating ETL processes using AWS services like Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker, Apache Spark.
  • Developing and maintaining data pipelines to move and transform data from diverse sources into data warehouses or data lakes.
  • Ensuring data quality and integrity through validation, cleansing, and monitoring ETL processes.
  • Optimizing ETL workflows for performance, scalability, and cost efficiency within the AWS environment.
  • Troubleshooting and resolving issues related to data processing and ETL workflows.
  • Implementing and maintaining security measures and compliance standards for data pipelines and infrastructure.
  • Documenting ETL processes, data mappings, and system architecture.
  • Implementing security measures such as IAM roles and access controls.
  • Diagnosing and resolving issues related to AWS services, infrastructure, and applications.
  • Proficiency in Big data tool and AWS services: Including Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker, Apache Spark relevant to data storage and processing.

Key Skills:

  • 12+ years of experience in ETL development.
  • Proficiency in ETL tools such as Talend.
  • Experience with databases including Snowflake, Oracle, Amazon RDS (Aurora, Postgres), DB2, SQL Server, and Cassandra.
  • Knowledge of big data tools and AWS services including Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker, Apache Spark.
  • Strong SQL skills for querying databases and manipulating data.
  • Experience with data modeling tools like ArchiMate, Erwin, and Oracle Data Modeler (preferred).
  • Familiarity with scheduling tools such as Autosys, SFTP, and AirFlow (preferred).
  • Ability to implement security measures and compliance standards.

Salary (Rate): undetermined

City: undetermined

Country: USA

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Job Role: ETL developer with AWS Cloud

Location: Remote

Experience: 12+ Years

ETL Tools:

Talend

Database:

Snowflake, Oracle, Amazon RDS (Aurora, Postgres), DB2, SQL server and Casandra,

Big Data and Amazon Services:

Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker, Apache Spark

Data Modeling Tools:

ArchiMate (not mandated- secondary/preferred), Erwin, Oracle Data Modeler (secondary/preferred)

Scheduling Tools: Autosys, SFTP, AirFlow (preferred. This should not be an issue, any resource can learn how to use it)

Key Responsibilities:

Designing, building, and automating ETL processes using AWS services like Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker Apache Spark.

Developing and maintaining data pipelines to move and transform data from diverse sources into data warehouses or data lakes.

Ensuring data quality and integrity through validation, cleansing, and monitoring ETL processes.

Optimizing ETL workflows for performance, scalability, and cost efficiency within the AWS environment.

Troubleshooting and resolving issues related to data processing and ETL workflows.

Implementing and maintaining security measures and compliance standards for data pipelines and infrastructure.

Documenting ETL processes, data mappings, and system architecture.

Implementing security measures such as IAM roles and access controls.

Diagnosing and resolving issues related to AWS services, infrastructure, and applications.

Proficiency in Big data tool and AWS services: Including Apache Sqoop, AWS S3, Hue, AWS CLI, Amazon EMR, Amazon MSK, Amazon Sagemaker, Apache Spark relevant to data storage and processing.

Strong SQL skills:

For querying databases and manipulating data during the transformation process.

Apply
Similar Jobs
Loading data...