AWS Data Engineer
Posted Today by 1764391623
Negotiable
Outside
Remote
USA
Summary: The AWS Data Engineer will be responsible for building and maintaining secure, scalable, and automated data pipelines to facilitate data ingestion, transformation, and curation. The role emphasizes the implementation of CI/CD pipelines, automated testing, and monitoring solutions to ensure data product integrity and performance. Collaboration with data modelers and architects is essential to align technical solutions with business requirements. The position requires a strong background in AWS services and data engineering practices.
Key Responsibilities:
- Develop configuration-driven ingestion and transformation pipelines using AWS Glue, Lambda, and Redshift.
- Implement CI/CD pipelines for automated build, test, and deployment of data products.
- Establish automated test frameworks for schema validation, data quality, and performance testing.
- Set up monitoring and observability dashboards for product-level metrics (CloudWatch, custom metrics).
- Work with data modelers and architects to ensure alignment with business requirements.
Key Skills:
- Proficiency in Python, SQL, and shell scripting.
- Hands-on experience with AWS services (Glue, Redshift, S3, Lambda, CloudWatch).
- Experience with CI/CD tools (Terraform, Jenkins, AWS CodePipeline).
- Strong knowledge of ETL/ELT processes and frameworks.
- Strong problem-solving, collaboration, and communication skills.
- Bachelor's degree in Computer Science, Data Engineering, or related field.
- 5+ years of experience in data engineering and pipeline development.
- AWS Certified Data Analytics or AWS Certified Developer is a plus.
Salary (Rate): £80/hr
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Position :- AWS Data Engineer
TechM193038
Role Summary
The Data Engineer will build and maintain secure, scalable, and automated data pipelines to support the ingestion, transformation, and curation of data products. This role will focus on implementing CI/CD pipelines, automated testing, and monitoring solutions.
Key Responsibilities
Data Pipelines: Develop configuration-driven ingestion and transformation pipelines using AWS Glue, Lambda, and Redshift.
Automation: Implement CI/CD pipelines for automated build, test, and deployment of data products.
Testing: Establish automated test frameworks for schema validation, data quality, and performance testing.
Monitoring: Set up monitoring and observability dashboards for product-level metrics (CloudWatch, custom metrics).
Collaboration: Work with data modelers and architects to ensure alignment with business requirements.
Required Skills
Programming: Proficiency in Python, SQL, and shell scripting.
AWS Expertise: Hands-on experience with AWS services (Glue, Redshift, S3, Lambda, CloudWatch).
Automation: Experience with CI/CD tools (Terraform, Jenkins, AWS CodePipeline).
Data Integration: Strong knowledge of ETL/ELT processes and frameworks.
Soft Skills: Strong problem-solving, collaboration, and communication skills.
Qualifications
Bachelor s degree in Computer Science, Data Engineering, or related field.
5+ years of experience in data engineering and pipeline development.
AWS Certified Data Analytics or AWS Certified Developer is a plus.
The pay range for this role is $140k - $145k per annum including any bonuses or variable pay/ 75-80/hr Tech Mahindra also offers benefits like medical, vision, dental, life, disability insurance and paid time off (including holidays, parental leave, and sick leave, as required by law). Ask our recruiters for more details on our Benefits package. The exact offer terms will depend on the skill level, educational qualifications, experience, and location of the candidate.