Senior Data Engineer - AWS/Databricks

Contract: Initial 12 months

Rate: Up to £475 per day - Inside IR35

Location: Birmingham

Working Pattern: Hybrid - 3 days per week onsite at Birmingham office

Key Responsibilities

• Design, build and maintain high-throughput data pipelines using Apache Spark and cloud-native technologies.
• Develop scalable ETL/ELT workflows for structured and unstructured datasets.
• Build and optimise data-processing solutions within AWS environments.
• Develop performant solutions using Java or Scala alongside advanced SQL.
• Support modern data lake and lakehouse architectures.
• Work alongside AI/ML teams to provide data platforms supporting Generative AI, machine learning and advanced analytics use cases.
• Implement appropriate data quality, governance, security and monitoring standards.
• Troubleshoot complex data-processing and distributed-system issues within production environments.

Essential Experience

• 5+ years of commercial Data Engineering experience.
• Strong hands-on Apache Spark development experience.
• Strong programming experience with Java and/or Scala.
• Strong AWS experience, ideally including technologies such as S3, EMR, Glue and IAM.
• Advanced SQL development skills.
• Strong experience designing and developing ETL/ELT pipelines.
• Experience working with large-scale and distributed data-processing environments.
• Databricks and/or Snowflake.