Data Engineer
Posted Today by Haystack
£65,000 Per year
Undetermined
Remote
London, England, United Kingdom
Summary: The Data Engineer role involves taking ownership of end-to-end data pipelines within a Palantir Foundry ecosystem for a leading UK infrastructure and facilities management company. The position requires expertise in Python, PySpark, and SQL to transform complex datasets for decision-making. The role also includes designing scalable data workflows and managing existing ETL processes while ensuring data integrity. This is an opportunity to work in a fully remote environment with significant career progression potential.
Key Responsibilities:
- Develop high-performance data pipelines and transformations using Python (Pandas) and PySpark within the Palantir Foundry environment.
- Design and implement scalable, reusable data workflows for advanced analytics and machine learning use cases.
- Lead collaborative workshops with stakeholders to capture requirements and translate them into robust ontology designs and data solutions.
- Manage day-to-day BAU tasks maintaining existing ETL processes while contributing to strategic roadmap projects and technology migrations.
- Ensure data integrity and performance by monitoring pipelines and optimizing processes to meet stringent enterprise SLAs.
Key Skills:
- Proven expertise in Palantir Foundry development, including extensive use of Code Workbooks, Pipeline Builder, and Object Explorer.
- At least 5 years of advanced programming experience specializing in Python, PySpark, and complex SQL scripting.
- Strong background in data modeling and ontology development to optimize data accessibility for reporting and AI workflows.
- Practical experience in automating ETL processes and working within CI/CD frameworks for seamless code deployment.
- Experience navigating multi-cloud environments (Azure/AWS) and managing data migrations from legacy systems into modern platforms.
Salary (Rate): £65,000 yearly
City: London
Country: United Kingdom
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Data Engineer | £50,000 - £65,000 We're working with a leading UK-wide infrastructure and enterprise facilities management powerhouse on this exciting opportunity. Join a dynamic data squad and take full ownership of end-to-end data pipelines within a sophisticated Palantir Foundry ecosystem. You will be at the heart of transforming complex datasets using Python, PySpark, and SQL to drive high-stakes decision-making across a massive enterprise landscape.
The Role
- Develop high-performance data pipelines and seamless transformations using Python (Pandas) and PySpark within the Palantir Foundry environment.
- Design and implement scalable, reusable data workflows that curate enterprise-level data for advanced analytics and machine learning use cases.
- Lead collaborative workshops with stakeholders to capture requirements and translate them into robust ontology designs and data solutions.
- Manage day-to-day BAU tasks maintaining existing ETL processes while contributing to strategic roadmap projects and technology migrations.
- Ensure data integrity and performance by monitoring pipelines and optimizing processes to meet stringent enterprise SLAs.
What You'll Need
- Proven expertise in Palantir Foundry development, including extensive use of Code Workbooks, Pipeline Builder, and Object Explorer.
- At least 5 years of advanced programming experience specializing in Python, PySpark, and complex SQL scripting.
- Strong background in data modeling and ontology development to optimize data accessibility for reporting and AI workflows.
- Practical experience in automating ETL processes and working within CI/CD frameworks for seamless code deployment.
- Experience navigating multi-cloud environments (Azure/AWS) and managing data migrations from legacy systems into modern platforms.
What's On Offer
- £50,000 - £65,000 base salary + £5,200 annual car allowance.
- Performance-linked bonus scheme (5% - 15%) and comprehensive pension plan.
- Fully remote working model with all high-end equipment provided.
- Significant career progression opportunities within a major market-leading organization.
Apply via Haystack today!