The role
Design and productionise scalable, high-throughput distributed data pipelines
Onboard and normalise diverse data sources into consistent, queryable schemas
Build the core data foundations used by AI/ML models and agents
Develop a knowledge layer allowing AI agents to read and write data with full traceability back to source
Work with complex IoT, geospatial, time-series and maritime data
Build secure, high-performance production systems using Python
Help shape technical and architectural decisions in a genuinely greenfield environment
What we’re looking for
Strong production-level Python
Experience building distributed data pipelines at scale
Strong understanding of data modelling, schema design and data lineage/provenance
Solid software engineering fundamentals
Experience taking data systems from prototype production
SC Clearance or willingness/eligibility to undergo clearance
Highly desirable experience
- Kafka / Flink or other streaming technologies
- Spark, Iceberg, Trino and modern lakehouse architectures
- Knowledge graphs / graph databases
- ML/AI platform integration
- Entity resolution / data fusion
- Geospatial or time-series data
- Docker / Kubernetes
- Restricted or air-gapped environments
This is an opportunity to work on a technically challenging, high-priority programme where the data platform will sit directly underneath sophisticated AI and optimisation capabilities.