Senior Data Engineer - Active DV Required
Posted Today by Matchtech
£78 Per hour
Outside
Onsite
Worcestershire
Summary: The role of Senior Data Engineer requires candidates with active DV clearance to design, implement, and maintain secure data pipelines using the Elastic Stack and Apache NiFi. This fully onsite position is with a leading defence and security company and involves ensuring compliance with strict security standards. The successful candidate will also collaborate with data architects and security teams to manage large-scale data flows and develop robust data models. The contract duration is initially set for six months.
Key Responsibilities:
- Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.
- Implement data ingestion, transformation, and integration processes, ensuring data quality and security.
- Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards.
- Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity.
- Develop robust data models to support analytics and reporting within secure environments.
- Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack.
- Build dashboards and visualizations in Kibana to enable data-driven decision-making.
- Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies.
- Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.
Key Skills:
- Active DV clearance.
- Expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana).
- Experience with Apache NiFi.
- Strong understanding of data ingestion, transformation, and integration processes.
- Ability to collaborate with data architects and security teams.
- Experience in managing large-scale data flows and ensuring data integrity.
- Proficiency in developing data models for analytics and reporting.
- Skills in troubleshooting and performance tuning of data pipelines.
- Experience in building dashboards and visualizations in Kibana.
- Knowledge of backup and disaster recovery strategies.
- Strong documentation skills for data architecture and workflows.
Salary (Rate): £78/hour
City: Worcester
Country: United Kingdom
Working Arrangements: on-site
IR35 Status: outside IR35
Seniority Level: Senior
Industry: IT
Location: Worcester area (fully onsite)
Duration: 6 month initial contract
Rate: £78ph LTD (Outside IR35)
Due to the nature of the role, candidates must hold active DV clearance.
Role details: Our client, a leading defence and security company, are looking for DV cleared Data Engineers to join their team on a contract basis. This is a fully onsite role with the option of compressed hours.
This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations.
Responsibilities not limited to:
- Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.
- Implement data ingestion, transformation, and integration processes, ensuring data quality and security.
- Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards.
- Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity.
- Develop robust data models to support analytics and reporting within secure environments.
- Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack.
- Build dashboards and visualizations in Kibana to enable data-driven decision-making.
- Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies.
- Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.
Interested? Apply today via the link provided!