All Jobs Vacancy

Senior Data Engineer

Posted 1 day ago by TechNET

KEY RESPONSIBILITIES

  • Analyse, profile and onboard raw manufacturing and operational data from external site systems.
  • Design, develop and maintain scalable Python-based ETL/ELT pipelines and Databricks notebooks using Apache Spark and PySpark.
  • Build reusable Bronze and Silver data layers using lakehouse architecture and asset-centric data models.
  • Develop data models and master data frameworks covering products, materials, equipment, processes and manufacturing batches.
  • Build data transformation and aggregation frameworks to support manufacturing KPIs, including yield, cycle time and process performance.
  • Implement automated data-quality checks, validation, reconciliation, monitoring and exception handling.
  • Optimise pipeline performance, scalability, reliability and reusability.
  • Establish data governance, metadata, lineage, security and engineering standards.
  • Collaborate with manufacturing, quality, analytics, business and technical stakeholders to translate requirements into scalable data solutions.
  • Follow Agile and DevOps practices, including Git-based version control, CI/CD, automated testing and production support.
  • Provide technical guidance and mentoring while promoting engineering best practices.

ESSENTIAL REQUIREMENTS

  • Bachelor's degree in Computer Science, Data Engineering, Information Technology, Engineering or a related discipline.
  • 7+ years of professional Data Engineering experience, including ownership of complex data engineering deliveries.
  • Advanced hands-on experience with Python and SQL.
  • Strong practical experience with Databricks, Apache Spark, PySpark and notebook-based development.
  • Experience designing and developing scalable ETL/ELT pipelines and data lakehouse solutions.
  • Knowledge of Bronze and Silver data architecture, Delta Lake or equivalent lakehouse patterns.
  • Strong understanding of data modelling, master data, dimensional modelling and data aggregation.
  • Experience implementing data-quality frameworks, metadata, data lineage and reconciliation.
  • Experience with Git, CI/CD, automated testing and Agile delivery.
  • Strong analytical, problem-solving and stakeholder-management skills.

DESIRABLE EXPERIENCE

  • Experience working with manufacturing, industrial or operational data environments.
  • Exposure to Manufacturing Execution Systems (MES), SAP, industrial historians or laboratory and quality systems.
  • Understanding of manufacturing processes, batch traceability, product and material data.
  • Experience developing manufacturing KPIs, including yield, cycle time, process deviations and operational performance metrics.
  • Experience working in regulated or validated data environments.
  • Experience with Azure, AWS or other enterprise cloud data platforms.
  • Knowledge of data observability, monitoring and production support practices.

APPLICATION PROCESS

Please submit your CV, clearly stating your current UK right-to-work status and availability.

Highlight your hands-on experience with Python, Databricks, PySpark and scalable data engineering solutions.

We are currently reviewing candidate profiles against the requirements of the role.

Candidates whose skills and experience align with the position will be contacted.

Thank you for your interest and patience.

Rate:
Not specified
Location:
United Kingdom
IR35 Status:
Not specified
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Senior

Take-Home Pay

Not Available

Visit calculators for additional details

Share job