Data Engineer

Data Engineer

Posted 5 days ago by Harnham

Apply
£650 Per day
Inside
Hybrid
City Of London, England, United Kingdom
Apply
Artificial Intelligence Automation Azure Databricks Cloud Computing Cloud Migration Cloud Technology Databricks Data Engineering Data Migration Data Visualisation Extract Transform Load (ETL) Investments Microsoft Azure Pandas (Python Package) Pyspark Python (Programming Language) Stakeholder Management Terraform Workflows

Summary: The Data Engineer role at a rapidly growing investment firm involves designing and implementing data pipelines, migrating legacy systems to Databricks, and collaborating with stakeholders to deliver tailored data solutions. The position is part of a skilled analytics team focused on modernizing infrastructure for advanced analytics and automation. This contract role is hybrid, requiring onsite presence in London two days a week. The firm is committed to becoming more AI- and data-enabled, providing an exciting opportunity for contributions to AI readiness.

Key Responsibilities:

  • Develop and maintain ETL pipelines, including manual and semi-manual data loads
  • Connect and integrate diverse data sources across cloud platforms
  • Collaborate with analytics and design teams to create bespoke, scalable data solutions
  • Support data migration efforts from Azure to Databricks
  • Use Terraform to manage and deploy cloud infrastructure
  • Build robust data workflows in Python (e.g., pandas, PySpark)
  • Ensure the platform is scalable, efficient, and ready for future AI use cases

Key Skills:

  • Strong experience with Azure and Databricks environments
  • Advanced Python skills for data engineering (pandas, PySpark)
  • Proficiency in designing and maintaining ETL pipelines
  • Experience with Terraform for infrastructure automation
  • Track record of working on cloud migration projects, especially Azure to Databricks
  • Comfortable working onsite in London 2 days/week and engaging cross-functionally
  • Strong communication and problem-solving abilities

Salary (Rate): £650.00/daily

City: City Of London

Country: United Kingdom

Working Arrangements: hybrid

IR35 Status: inside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

DATA ENGINEER - INVESTMENT FIRM 6-MONTH CONTRACT £600 PER DAY INSIDE IR35 HYBRID

ABOUT THE COMPANY

A rapidly growing investment firm headquartered in London, which has doubled its team over the past 2.5 years and continues to attract strong investor backing. With 230+ employees and ongoing expansion, the company is now focused on becoming more AI- and data-enabled, modernising its infrastructure to support more advanced analytics and automation.

THE TEAM

You'll join a skilled and collaborative analytics team of eight, working closely with senior leadership. The team has already laid the foundations for a modern data platform using Azure and Databricks and is now focused on building out scalable ETL processes, integrating AI tools, and delivering bespoke analytics solutions across the organisation.

THE ROLE

As a Data Engineer, you'll play a pivotal role in designing and implementing robust data pipelines, supporting the migration from legacy Azure systems to Databricks, and working closely with stakeholders to deliver tailored data solutions. This role combines hands-on development with collaborative architecture design, and offers the opportunity to contribute to AI readiness within a fast-paced business.

KEY RESPONSIBILITIES

  • Develop and maintain ETL pipelines, including manual and semi-manual data loads
  • Connect and integrate diverse data sources across cloud platforms
  • Collaborate with analytics and design teams to create bespoke, scalable data solutions
  • Support data migration efforts from Azure to Databricks
  • Use Terraform to manage and deploy cloud infrastructure
  • Build robust data workflows in Python (e.g., pandas, PySpark)
  • Ensure the platform is scalable, efficient, and ready for future AI use cases

REQUIRED SKILLS & EXPERIENCE

  • Strong experience with Azure and Databricks environments
  • Advanced Python skills for data engineering (pandas, PySpark)
  • Proficiency in designing and maintaining ETL pipelines
  • Experience with Terraform for infrastructure automation
  • Track record of working on cloud migration projects, especially Azure to Databricks
  • Comfortable working onsite in London 2 days/week and engaging cross-functionally
  • Strong communication and problem-solving abilities

NICE TO HAVES

  • Experience with Qlik or other data visualisation tools
  • Exposure to AI product integration or readiness projects
Apply
Similar Jobs
Loading data...