Technical Manager (Data Engineering)
Posted 1 day ago by Thrive IT Systems
Negotiable
Undetermined
Undetermined
London Area, United Kingdom
Summary: The Technical Manager (Data Engineering) role focuses on leading the design, development, and delivery of cyber data platform solutions. The position requires strong technical leadership, expertise in data architecture, and proficiency in various technologies including Python, Databricks, and Apache Kafka. The manager will also ensure data security and compliance while implementing automation and DevOps practices. The ideal candidate will have experience in building resilient data platforms and guiding teams in delivering optimal product solutions.
Key Responsibilities:
- Lead the design, development, and delivery of cyber data platform solutions.
- Provide technical guidance and mentorship across all teams of the cyber data platform.
- Design, build, and manage real-time, near real-time, and batch data architectures.
- Contribute to the quality of the team's codebase through high-quality coding and peer reviews.
- Implement and enforce data security best practices and compliance with regulations.
- Implement automation and DevOps practices for data platform components.
Key Skills:
- Strong proficiency in Python and SQL.
- Experience with Databricks, Apache Kafka, Airflow, and dbt.
- Knowledge of Kubernetes and containerization technologies like Docker.
- Expertise in ETL and ELT frameworks for large-scale data processing.
- Understanding of cybersecurity principles and practices.
- Experience in leading teams and providing technical guidance.
Salary (Rate): undetermined
City: London Area
Country: United Kingdom
Working Arrangements: undetermined
IR35 Status: undetermined
Seniority Level: undetermined
Industry: Other
Job Description
Must have Skills : Python (Strong), Databricks, Apache Kafka, Airflow/DBT
Good To Have Skills : Kubernetes (Strong)
Technical Leadership: Take in charge of the design, development and delivery of cyber data platform solutions and provide technical guidance and mentorship across all teams of cyber data platform.
Data Architecture: Design, build and management of realtime, near realtime and batch data architectures that support threat detection, incident response and reporting through advanced analytics, machine learning and GenAI capabilities.
Coding: Contribute to raising the quality bar of the team''s codebase by producing high-quality code, conducting thorough peer reviews, and proactively providing constructive feedback to other team members on their code
Data Security and Compliance: Implement and enforce data security standard methodologies, ensuring the protection of sensitive information and compliance with relevant regulations.
Automation and DevOps: Implement various automations and DevOps practices to streamline the deployment, configuration, and management of data platform components.
You will need Proven experience in leading the teams to deliver resilient systems and providing technical guidance for optimal product solutions.
Experience building data platforms on cloud services like Databricks on Azure or GCP BigQuery, with the goal of delivering a self-service data platform for users.
Proficient in Python and SQL, with a solid grasp of architectural patterns, coding standards, code reviews, version control, and CI/CD practices.
Expertise in ETL and ELT frameworks for large scale realtime and batch data processing, with hands-on experience in Kafka, Flink, Airflow and dbt as well as containerisation technologies like Docker and Kubernetes.
Ability to provide clear input, guide, opportunities to help engineers develop and advance.
Knowledge of cybersecurity principles and practices.
Recruiter's Email : shikharsharma@thriveitsystems.com