Summary: The role of Senior Data Engineer involves hands-on responsibilities within the financial services sector, focusing on full-stack data engineering and transitioning towards AI/ML engineering. The position requires expertise in Databricks and the ability to design and implement data pipelines for unstructured data. The engineer will collaborate with stakeholders to enhance data capabilities and support the existing data team. This role is hybrid, requiring three days on-site in London.

Key Responsibilities:

• Industrialise the vectorisation of documents to support the implementation of RAG
• Utilising MosaicAI within Databricks on Azure
• Data migration from Blob Storage to Databricks
• Design and implement pipelines for ingesting unstructured data into Mosaic
• Populating an existing vector database
• Support the existing data team in overcoming bottlenecks and advancing their data capabilities
• Collaborate with stakeholders to ensure solutions align with business requirements
• Work across Dev and UAT workspaces, with a focus on scalability and performance
• Leverage experience with vector databases (eg, Quad Vector DB) to enhance data solutions
• Transition from traditional data engineering to AI/ML engineering as required

Key Skills:

• Strong work history as a Senior Data Engineer with full-stack data engineering capabilities
• Expert in Databricks (including productionizing)
• Proficiency in working with structured & unstructured data, including video & text data
• Strength in creation of data ingestion pipelines
• Familiarity with Azure cloud services and enterprise data platforms

Salary (Rate): undetermined

City: London

Country: UK

Working Arrangements: hybrid

IR35 Status: inside IR35

Seniority Level: Senior

Industry: Finance