Data Engineer

Your new company

You'll be working with IBM on an urgent public-sector programme, supporting the delivery of enterprise-scale data platforms and solutions.

Your new role

As a Data Engineer, you'll work across the data life cycle, with a strong focus on data ingestion, transformation, migration, warehousing and data management.

You'll be working with AWS data technologies alongside IBM DataStage, helping to build and maintain robust data integration and transformation processes across large-scale data environments.

Key responsibilities

• Design, develop and maintain enterprise-scale data ingestion and transformation processes.
• Develop and support ETL and data migration pipelines across both batch and Real Time processing.
• Support the development and management of large-scale enterprise data warehouse solutions.
• Build and maintain data integration and transformation processes using IBM DataStage.
• Work with AWS technologies including Amazon Redshift, Amazon S3, AWS Glue and Amazon QuickSight.
• Develop and optimise SQL-based data transformations and work with relational databases.
• Support database storage, management and wider data-platform activities.
• Apply DevSecOps principles to data engineering and deployment processes.
• Use GitHub/GitLab for source control and collaborative engineering.
• Contribute to Java development where required within the wider data platform.

What you'll need to succeed

You'll need strong commercial data engineering experience, particularly across enterprise-scale ingestion, transformation, warehousing and database environments.

Essential skills and experience

• IBM DataStage
• Amazon Redshift
• Amazon S3
• AWS Glue
• Amazon QuickSight
• AWS data/cloud environments
• ETL and data migration
• Batch and Real Time data processing
• Enterprise data warehouse development
• Large-scale data ingestion and transformation
• SQL
• Relational databases
• Database/DBA knowledge
• DevSecOps
• Java
• GitHub/GitLab
• Minimum 5 years' continuous/unbroken UK residency