Location: Hybrid (2 days in office, Glasgow)

Day Rate: 390/day (Inside IR35)

About the Role:

We are seeking an experienced Data Engineer to join our blue-chip client s dynamic team. The ideal candidate will have extensive hands-on experience in data engineering, particularly with Spark, Python, and Java. A background in banking or financial services is highly desirable, as you will work on projects that require domain-specific knowledge. If you have a deep understanding of big data ecosystems, enjoy solving complex problems, and thrive in a hybrid work environment, this role is for you!

Key Responsibilities:

• Develop and optimize scalable ETL pipelines using Spark and other big data technologies.


• Work on large datasets using HDFS, Hive, and similar tools in the big data ecosystem.


• Implement and maintain CI/CD pipelines to automate data workflows.


• Collaborate with cross-functional teams to design data solutions and support ongoing analytics efforts.


• Ensure data quality, availability, and performance in production environments.

Core Skills & Experience Required:

• Minimum of 8+ years of experience in data engineering.


• Hands-on experience with Spark 2.x/3.x.


• Proficiency in Python (or Scala/Java), with a preference for Python.


• Strong knowledge of HDFS, SQL, and experience in building ETL pipelines.


• Familiarity with big data ecosystems including data nodes/edge nodes, Hive, and similar technologies.


• Experience in data wrangling and working with large datasets.


• Competency with CI/CD pipelines and experience in deploying scalable data solutions.

Desirable:

• Previous experience in banking or financial services industries.


• Familiarity with financial data structures and regulatory requirements.