Key Responsibilities
- Design and maintain end-to-end Azure data pipelines, from ingestion through to production use.
- Work with structured, semi-structured and unstructured data at scale.
- Build solutions using Azure Cosmos DB, Synapse and Databricks.
- Apply SQL and NoSQL data modelling and optimisation techniques.
- Support GenAI use cases including RAG, embeddings, search and retrieval.
- Prepare and transform data for production AI systems.
- Perform data discovery, profiling, transformation and quality analysis.
- Work with data scientists, AI engineers, architects and business stakeholders to deliver scalable solutions.
Essential Experience
- Strong, hands-on Azure Data Engineering experience.
- Proven experience building and maintaining production-grade data pipelines.
- Strong experience with Azure Cosmos DB and NoSQL concepts.
- Experience with Azure Synapse.
- Strong conceptual understanding of Azure Databricks.
- Strong SQL and Python skills.
- Experience with high-volume, structured/unstructured data.
- Practical experience contributing to a live/real-world GenAI use case - not purely theoretical.
- Understanding of how data is prepared and integrated into production AI/GenAI systems.
- Familiarity with CI/CD, DevOps and modern data architectures.
The Ideal Candidate
You'll combine strong data engineering fundamentals, deep Azure knowledge and practical GenAI experience.
You should be able to explain not only how to build a data pipeline, but how the underlying data, SQL/NoSQL platforms and architecture support AI in production.
Azure experience is essential - candidates with strong general data engineering skills but limited Azure data-service exposure are unlikely to be suitable.
Rate:
£650/day
Location:
London
IR35 Status:
Inside
Remote Status:
Hybrid
Industry:
Data & Analytics
Seniority Level:
Not Specified