Senior AWS Data engineer (LDW Data Warehouse Discovery)
Posted Today by Experis IT
£483 Per day
Inside
Hybrid
Telford, UK
Summary: The Senior AWS Data Engineer role focuses on migrating data from legacy systems to a new AWS cloud-native architecture within the Data Contract Delivery Area. The position requires collaboration with an Agile software delivery team and involves designing and delivering cloud transformation projects. Candidates should possess strong AWS skills and experience in data engineering, particularly in creating data pipelines and implementing ETL/ELT processes. The role also emphasizes the importance of DevOps practices and stakeholder interaction.
Key Responsibilities:
- Support the technical lead with design and client interactions, and assist junior engineers with their development.
- Design, develop, and test data pipelines to ensure data readiness for analytics and reporting.
- Implement ETL/ELT processes to move data from source systems to Data Warehouses/Data Lakes/Lake Houses using Open Source and AWS tools.
- Utilize DevOps methodologies and tools for continuous integration and deployment (CI/CD).
Key Skills:
- Proficiency with Core AWS Tools (AWS Glue, Lambda, S3, Redshift).
- Programming Skills (Python).
- SQL and Data Storage Technologies knowledge (AWS Redshift, AWS RDS).
- Experience with AWS data lakes on AWS S3.
Salary (Rate): £483 daily
City: Telford
Country: UK
Working Arrangements: hybrid
IR35 Status: inside IR35
Seniority Level: Senior
Industry: IT
Detailed Description From Employer:
Senior AWS Data engineer (LDW Data Warehouse Discovery)
Max Supplier Rate: £483
Clearance Required: SC ACTIVE
Duration: 6 months
Location: Telford with 2 days/week in office
IR35 Status: Inside
The role falls within the Data Contract Delivery Area of the clients contract. The group provides a wide range of data and analytics solutions in support of our client's business priorities: maximise revenues, bear down on fraud, and cloud migration.
This role involves migrating data from Legacy on-premise systems (primarily Oracle and Informatica) to a new AWS cloud-native architecture.
You will be part of an Agile software delivery team working closely with other engineers and supported by project managers, business analysts and architects. With additional client and key stakeholder interaction as required.
We are looking for strong AWS Senior Data Engineers who can design and deliver cloud transformation projects.
Your work will be to:
- As part of a cloud transformation team, supporting the technical lead with design and client interactions, and supporting junior engineers with their development.
- Design, Develop and Test Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.
- Implement ETL/ELT Processes: Develop and Test Extract, Transform, Load (ETL) or Extract, Load, Transform (ELT) workflows to seamlessly move data from source systems to Data Warehouses/Data Lakes/Lake Houses using Open Source and AWS tools.
- Adopt DevOps Practices: Utilise DevOps methodologies and tools for continuous integration and deployment (CI/CD).
Must-have skills:
- Proficiency with Core AWS Tools (AWS Glue, Lambda, S3, Redshift)
- Programming Skills (Python)
- SQL and Data Storage Technologies: Some knowledge of Data Warehouse, Database technologies, and technologies (AWS Redshift, AWS RDS).
- AWS Data Lakes: Some experience with AWS data lakes on AWS S3 to store and process both structured and unstructured data sets.
Nice-to-have skills:
- Knowledge of Open Table Formats (Iceberg/Delta).
- AWS Tools: Experience with Amazon CloudWatch, SNS, Athena, DynamoDB, EMR, Kinesis.
- Data modelling
- Job scheduling/orchestration
- Data virtualisation tools (Denodo)
- ALM Tooling (Jira, Confluence)
- CI/CD toolsets (GitLab, Terraform)
- Reporting tools (Business Objects, Power BI, Pentaho BA)
- Data Analytics toolset (SAS Viya)
- Observability tools (Grafana, Dynatrace)
Experience:
You should have experience as a senior data engineer delivering within large scale data analytics solutions and the ability to operate at all stages of the software engineering life cycle, as well as some experience in the following
- Awareness of DevOps culture and modern engineering practices
- Experience of Agile Scrum based delivery
- Proactive in nature, personal drive, enthusiasm, willingness to learn
- Excellent communications skills including stakeholder management
- Developing solutions within the given architecture and adhering to specified NFRs
- Supporting other engineers within your team
- Continually looking for ways to improve