AWS Data Engineer
Posted 7 days ago by N Consulting Global
Negotiable
Undetermined
Hybrid
London Area, United Kingdom
Summary: The AWS Data Engineer role at FTSE Russell involves designing and developing scalable data pipelines and workflows using AWS tools and modern software engineering practices. The position requires collaboration with business teams to create data-driven solutions and contribute to a lakehouse architecture. Candidates should have a strong background in Python and data engineering, with a willingness to learn and adapt to new technologies. This is a hybrid fixed-term contract position based in London, UK.
Key Responsibilities:
- Designing and developing scalable, testable data pipelines using Python and Apache Spark
- Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3
- Applying modern software engineering practices: version control, CI/CD, modular design, and automated testing
- Contributing to the development of a lakehouse architecture using Apache Iceberg
- Collaborating with business teams to translate requirements into data-driven solutions
- Building observability into data flows and implementing basic quality checks
- Participating in code reviews, pair programming, and architecture discussions
- Continuously learning about the financial indices domain and sharing insights with the team
Key Skills:
- Proficient in writing clean, maintainable Python code
- Understanding of data engineering basics: batch processing, schema evolution, and building ETL pipelines
- Experience with or eagerness to learn Apache Spark for large-scale data processing
- Familiarity with the AWS data stack (e.g. S3, Glue, Lambda, EMR)
- Ability to work well in Agile teams and value collaboration
- Nice-to-haves include experience with Apache Iceberg, CI/CD tools, and data quality frameworks
- Curiosity about financial markets, index data, or investment analytics
Salary (Rate): undetermined
City: London
Country: United Kingdom
Working Arrangements: hybrid
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Job title: AWS Data Engineer
Job location: London, UK
Job mode: Hybrid
Job type: FTC
Experience:10+ Years
We are building the next-generation data platform at FTSE Russell — and we want you to shape it with us.
Your role will involve:
- Designing and developing scalable, testable data pipelines using Python and Apache Spark
- Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3
- Applying modern software engineering practices: version control, CI/CD, modular design, and automated testing
- Contributing to the development of a lakehouse architecture using Apache Iceberg
- Collaborating with business teams to translate requirements into data-driven solutions
- Building observability into data flows and implementing basic quality checks
- Participating in code reviews, pair programming, and architecture discussions
- Continuously learning about the financial indices domain and sharing insights with the team
WHAT YOU'LL BRING:
- Writes clean, maintainable Python code (ideally with type hints, linters, and tests like pytest)
- Understands data engineering basics: batch processing, schema evolution, and building ETL pipelines
- Has experience with or is eager to learn Apache Spark for large-scale data processing
- Is familiar with the AWS data stack (e.g. S3, Glue, Lambda, EMR)
- Enjoys learning the business context and working closely with stakeholders
- Works well in Agile teams and values collaboration over solo heroics
Nice-to-haves: It’s great (but not required) if you also bring:
- Experience with Apache Iceberg or similar table formats
- Familiarity with CI/CD tools like GitLab CI, Jenkins, or GitHub Actions
- Exposure to data quality frameworks like Great Expectations or Deequ
- Curiosity about financial markets, index data, or investment analytics