All Jobs Vacancy

Data Engineer

Posted 1 day ago by Pyramid Consulting, Inc

What the contractor will do

Write production-grade Python and develop agents to automate R&D data processing.

The work needs to be tested, instrumented and documented well enough for the team to run long term.

Design data storage, transfer and transformation utilities, including secure inbound and outbound data exchange with automated quality control.

Work with ML engineers and software engineers to shape R&D datasets around what the models and the science need.

Parse and normalize metadata, validate incoming manifests against the agreed standard, and build quality and de-identification checks to replace manual review.

Required skills and experience

  • Strong technical or platform engineering and MLOps background, combined with GenAI automation and development
  • Strong Python for data engineering, including data transformation, job monitoring and schema management
  • Experience with large binary or blob datasets in cloud object storage (Google Cloud Storage, Azure Blob Storage/ADLS, Amazon S3 or equivalent)
  • Significant SQL experience, including schema design (BigQuery a plus)
  • Experience building and running ETL/ELT pipelines with an orchestration framework
  • Experience with CI/CD, agile development and DevOps
  • Ability to work independently with ambiguous requirements

Education/experience

  • PhD in a computational science or AI/ML with R&D data (any domain), or an MSc with 2+ years of industry experience
Rate:
Not specified
Location:
London
IR35 Status:
Not specified
Remote Status:
Hybrid
Industry:
Data & Analytics
Seniority Level:
Senior

Take-Home Pay

Not Available

Visit calculators for additional details

Share job