Principal Data Engineer – SC Cleared

Job Spec

The Principal Data Engineer is accountable for the design, implementation, governance, and continuous improvement of enterprise data platforms, pipelines, and architectures. This role provides technical leadership across data engineering activities, ensuring data solutions are secure, scalable, reliable, and aligned with business objectives.

The Principal Data Engineer leads the development of modern cloud-based data platforms, promotes engineering best practices, and establishes standards for data integration, data quality, data governance, and DevOps/DataOps principles. The role partners with business stakeholders, architects, analysts, and software engineering teams to enable data-driven decision-making and advanced analytics capabilities across the organisation.

Main Responsibilities

• Design and maintain scalable, secure, and resilient data architectures and platforms.
• Lead the development of enterprise data pipelines, data integration frameworks, and data products.
• Establish and enforce data engineering standards, patterns, and best practices.
• Implement DataOps and DevOps methodologies to improve deployment efficiency, automation, and operational reliability.
• Collaborate with cross-functional teams to understand data requirements and deliver effective solutions.
• Ensure data quality, integrity, lineage, security, and compliance across all data assets.
• Provide technical leadership, mentoring, and coaching to data engineering teams.
• Support cloud migration and modernization initiatives involving data platforms and services.
• Identify opportunities to optimize performance, reduce costs, and increase scalability.
• Manage technical risks, dependencies, and delivery challenges associated with data solutions.
• Evaluate emerging technologies and recommend innovative approaches to enhance data capabilities.
• Support development of data governance frameworks and architecture roadmaps.

Key Skills

• Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, Information Systems, or a related discipline.
• 7+ years of experience in data engineering, software engineering, or data platform development.
• Significant experience designing and implementing enterprise-scale data solutions.
• Strong understanding of cloud-based data platforms and modern data architectures.
• Strong expertise in designing and implementing enterprise-scale data architectures.
• Extensive experience with cloud data platforms such as Microsoft Azure, AWS, or Google Cloud.
• Strong proficiency in SQL, Python, Spark, and modern data processing frameworks
• Experience building and managing ETL/ELT pipelines and streaming data solutions.
• Practical experience with DataOps, DevOps, CI/CD pipelines, and Infrastructure as Code.
• Expertise in data modelling, data warehousing, and master data management principles.
• Strong understanding of data governance, metadata management, and data quality frameworks.
• Experience using tools such as Azure Data Factory, Databricks, Snowflake, Fabric, Airflow, or similar technologies
• Leadership skills with the ability to mentor engineers and influence technical direction
• Excellent stakeholder engagement and communication skills
• Strategic thinking and the ability to align technical solutions with business goals.
• Strong analytical and problem-solving capabilities.
• Experience working in Agile delivery environments and cross-functional teams.

Desirable

• Master’s degree in Computer Science, Data Science, Engineering, or a related field.
• Professional cloud certifications (AWS, Microsoft Azure, or Google Cloud).
• Certifications in Data Engineering, Data Management, or Enterprise Architecture.
• TOGAF, DAMA CDMP, or equivalent industry-recognised certifications.
• Experience working within aerospace, defence, manufacturing, or highly regulated industries
• Knowledge of AI/ML platforms and MLOps practices.
• Experience with graph databases and knowledge engineering solutions.
• Experience implementing data mesh or data fabric architectures.
• Enterprise Architecture experience.
• Experience leading large-scale data modernization programmes.
• Knowledge of defence security standards and classified environments.