The Opportunity
Primus Connect is working with a growing Data & AI Consultancy that is delivering modern data platforms and Databricks solutions for a range of enterprise clients.
They are expanding their engineering capability and are looking for a Mid-Level Data Engineer with strong commercial Databricks and Spark experience to join an active delivery team.
This is a hands-on engineering position where you will take ownership of data pipeline development, work with modern Databricks technologies and collaborate closely with senior engineers and wider delivery teams.
Key Responsibilities
- Design, build and maintain ETL/ELT data pipelines using Databricks, PySpark and Spark SQL
- Develop and optimise Delta Lake tables using Bronze, Silver and Gold medallion architecture
- Ingest data from a variety of source systems into Databricks
- Write clean, maintainable and well-tested Python/data engineering code
- Implement data quality checks, validation and monitoring
- Participate in code reviews and collaborative development
- Contribute to CI/CD pipelines for data engineering workloads
- Work with senior engineers to deliver scalable data platform solutions
- Collaborate with technical, analytics and business stakeholders
- Participate in Agile ceremonies and delivery activities
Essential Experience
- Commercial experience building data pipelines using Databricks
- Strong PySpark and Spark SQL skills
- Solid Python experience within data engineering
- Practical experience with Delta Lake
- Understanding of medallion architecture and modern data platform principles
- Experience working with Azure, AWS or GCP
- Good understanding of Git and collaborative development practices
- Experience working within Agile delivery environments
- Strong communication skills and the ability to work effectively in a remote team
Desirable Experience
- Databricks certification
- Unity Catalog
- Databricks Asset Bundles
- Terraform/Infrastructure as Code
- Databricks Workflows
- Airflow
- Spark Structured Streaming
- Kafka or Azure Event Hubs
Rate:
£525/day
Location:
United Kingdom
IR35 Status:
Outside
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Mid-Level