AWS Data Engineer
Posted 2 weeks ago by Sentinel
£55,000 Per year
Fixed-Term
Hybrid
York, North Yorkshire
Summary: The role of Data Engineer focuses on driving Machine Learning initiatives within a leading UK rail organization undergoing digital transformation. This 6-month fixed-term contract position requires building robust data pipelines and ETL processes to support predictive solutions across multiple train operating companies. The successful candidate will collaborate with internal teams to enhance operational efficiency and customer experience through innovative data solutions. This position is ideal for a mid-level professional with extensive experience in cloud-based data engineering.
Key Responsibilities:
- Provide technical support for data acquisition across multiple Train Operating Companies.
- Develop efficient data ingestion and ETL pipelines for multi-operator ML models.
- Manage and monitor the performance and reliability of ML pipelines.
- Mentor team members on best practices for efficient data ingestion and quality monitoring.
Key Skills:
- Extensive experience with cloud-based data ingestion pipelines (e.g., Glue, lambda, athena, Redshift, RDS, kinesis).
- Strong understanding of data ingestion, data modelling, data warehousing, and building scalable data pipelines.
- Significant professional experience as a Data Engineer on various in-house projects.
- Proficiency in SQL, Python, and cloud platforms (e.g., AWS, Big Query), with strong coding, tuning, testing, and troubleshooting skills.
- Expertise in data modelling and end-to-end development of tested ETL pipelines and infrastructure as code in a cloud environment.
- Proven experience leading medium to large-scale data warehousing and analytics using cloud technologies (e.g., Redshift, S3, EC2, RDS, Glue, Athena, and other big data tools).
- Excellent data management practice, including dataset versioning, dictionary keeping, and data documentation.
- Experience in deployment, execution, and operations of automated CI/CD data pipelines (bonus).
- Experience building distributed systems for data extraction, ingestion, and processing multiple resources (bonus).
- Familiarity with a Unix environment (bonus).
- Previous experience working with data scientists (desirable, but not essential).
Salary (Rate): 60000
City: London
Country: UK
Working Arrangements: hybrid
IR35 Status: fixed-term
Seniority Level: Mid-Level
Industry: IT