Data Platform Engineer
Posted Today by Primus Connect
£72 Per hour
Outside
Remote
London Area, United Kingdom
Summary: The Data Platform Engineer role focuses on building and optimizing data pipelines within a modern data platform/Lakehouse environment using GCP, PySpark, and Airflow. This hands-on engineering position requires collaboration with the data engineering team to ensure reliable, scalable, and performant systems. The role also involves managing workflows and supporting the underlying platform infrastructure. The position is contract-based, with a preference for remote work and occasional travel to London.
Key Responsibilities:
- Design and build end-to-end data pipelines (PySpark/SQL)
- Develop and manage workflows using Airflow (DAGs, scheduling, orchestration)
- Work with BigQuery for large-scale data processing and analytics
- Monitor, debug, and optimise pipelines for performance and reliability
- Collaborate with data analysts and stakeholders to validate outputs
- Support and improve GCP data platform infrastructure
- Build and manage Terraform (IaC) for scalable environments
- Implement robust data ingestion and transformation patterns
Key Skills:
- Strong experience as a Data Platform Engineer (GCP)
- Experience with PySpark/Apache Spark
- Strong experience with Airflow (or similar orchestration tools)
- Experience working with BigQuery
- Solid understanding of distributed data processing
- Experience building data pipelines end-to-end
- Experience with Terraform (modules, reusable infrastructure)
- Strong debugging and problem-solving skills
- Nice to Have: Experience with data quality tools (Dataplex, Great Expectations, Soda)
- Exposure to AI/ML or LLM-based workflows
- Experience with containerisation (Docker/Kubernetes)
Salary (Rate): £72.00/hr
City: London
Country: United Kingdom
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Data Platform Engineer (GCP, PySpark, Airflow) - Contract
Rate : £550 - £575 per day Outside IR35
Location : Remote (UK)
Travel : London once per month (preferred)
Duration : 3 months (initial)
Start : ASAP
The Role
We're hiring a Data Platform Engineer (GCP, PySpark, Airflow) to support a modern data platform/Lakehouse environment . This is a hands-on engineering role focused on building and optimising data pipelines , while also contributing to the underlying platform and infrastructure. You'll work closely with the data engineering team, delivering production-grade pipelines and ensuring systems are reliable, scalable, and performant.
Key Responsibilities
- Design and build end-to-end data pipelines (PySpark/SQL)
- Develop and manage workflows using Airflow (DAGs, scheduling, orchestration)
- Work with BigQuery for large-scale data processing and analytics
- Monitor, debug, and optimise pipelines for performance and reliability
- Collaborate with data analysts and stakeholders to validate outputs
- Support and improve GCP data platform infrastructure
- Build and manage Terraform (IaC) for scalable environments
- Implement robust data ingestion and transformation patterns
Required Skills
- Strong experience as a Data Platform Engineer (GCP)
- Experience with PySpark/Apache Spark
- Strong experience with Airflow (or similar orchestration tools)
- Experience working with BigQuery
- Solid understanding of distributed data processing
- Experience building data pipelines end-to-end
- Experience with Terraform (modules, reusable infrastructure)
- Strong debugging and problem-solving skills
Nice to Have
- Experience with data quality tools (Dataplex, Great Expectations, Soda)
- Exposure to AI/ML or LLM-based workflows
- Experience with containerisation (Docker/Kubernetes)
What We're Looking For
Engineers who can work independently and deliver quickly
Strong ownership mindset, able to troubleshoot and improve systems
Comfortable working across data and platform
Focused on delivery over over-engineering
Please apply with an up to date copy of your CV.