Senior Data Engineer
Posted Today by Webologix Ltd/ INC
Negotiable
Undetermined
Undetermined
Warwick, England, United Kingdom
Summary: The Senior Data Engineer will lead the architectural direction and technical leadership for the Interconnectors Data Platform (ICDP), a cloud-based data warehouse. This role involves designing scalable data architectures, migrating legacy ETL tools to modern Azure-based pipelines, and ensuring data quality and governance. The engineer will also mentor team members and collaborate with various stakeholders to deliver high-quality outcomes.
Key Responsibilities:
- Lead the design and implementation of scalable data architectures, including Bronze/Silver/Gold layered models.
- Drive the platform’s architectural roadmap and ensure alignment with modern engineering best practices.
- Develop robust, secure, and observable ingestion and transformation pipelines.
- Lead the migration from legacy ETL tools into modern Azure-based pipelines, including Azure Functions and Azure Data Factory.
- Build and maintain performant SQL-based transformations, curated layers, and reusable data models.
- Ensure CI/CD, testing, version control, and observability are fully embedded in workflows.
- Implement data validation, reconciliation, profiling, and auditability across platform layers.
- Work with business stakeholders to ensure output meets analytical and operational needs.
- Mentor data engineers and contribute to the growth of technical capability within the ICDP team.
- Collaborate with Product teams, IT&D, and external partners to deliver high-quality outcomes.
Key Skills:
- Strong, hands-on experience building production-grade data pipelines, transformations, and orchestration using Python.
- Expert-level analytical SQL, query optimization, and data modelling.
- Experience with Azure Functions, Azure Data Factory, Azure Storage, Key Vault, and cloud security fundamentals.
- Deep understanding of data architecture principles, dimensional modelling, and scalable enterprise data design.
- Strong grounding in Git, CI/CD, automated testing, branching strategies, and modern engineering practices.
- Experience with API ingestion, SFTP ingestion, and resilient pipeline design.
- Strong problem-solving and architectural thinking.
- Excellent communication and stakeholder collaboration skills.
- Ability to provide clarity and leadership in complex or ambiguous technical landscapes.
- Experience contributing to or leading data-platform re-architecture programs.
Salary (Rate): undetermined
City: Warwick
Country: United Kingdom
Working Arrangements: undetermined
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
The Role
The Interconnectors Data Platform (ICDP) is a cloud-based data warehouse enabling critical commercial, financial modelling, and operational decision-making. As the platform moves toward a modernized Medallion Architecture and Azure-native ingestion patterns, we are hiring a Senior Data Engineer to provide architectural direction and hands-on technical leadership.
Your responsibilities: (Up to 10, Avoid repetition)
- Data Architecture & Platform Engineering
- - Lead the design and implementation of scalable data architectures, including Bronze/Silver/Gold layered models.
- - Drive the platform’s architectural roadmap and ensure alignment with modern engineering best practices.
- - Develop robust, secure, and observable ingestion and transformation pipelines.
- Pipeline Development & Operations
- - Lead the migration from legacy ETL tools (e.g., Matillion) into modern Azure-based pipelines, including Azure Functions, Azure Data Factory (ADF), and event-driven frameworks.
- - Build and maintain performant SQL-based transformations, curated layers, and reusable data models.
- - Ensure CI/CD, testing, version control, and observability are fully embedded in workflows.
- Data Quality & Governance
- - Implement data validation, reconciliation, profiling, and auditability across platform layers.
- - Work with business stakeholders to ensure output meets analytical and operational needs.
- Leadership
- - Mentor data engineers and contribute to the growth of technical capability within the ICDP team.
- - Collaborate with Product teams, IT&D, and external partners to deliver high-quality outcomes.
- - Serve as a technical authority for engineering approaches, patterns, and standards.
Your Profile
Essential skills/knowledge/experience:
Essential Technical Skills
- - Python : Strong, hands-on experience building production-grade data pipelines, transformations, and orchestration using Python.
- - Advanced SQL: Expert-level analytical SQL, query optimization, and data modelling.
- - Azure Cloud: Experience with Azure Functions, Azure Data Factory, Azure Storage, Key Vault, and cloud security fundamentals.
- - Data Warehousing: Deep understanding of data architecture principles, dimensional modelling, and scalable enterprise data design.
- - Strong grounding in Git, CI/CD, automated testing, branching strategies, and modern engineering practices.
- - Experience with API ingestion, SFTP ingestion, and resilient pipeline design.
Soft Skills
- - Strong problem-solving and architectural thinking.
- - Excellent communication and stakeholder collaboration skills.
- - Ability to provide clarity and leadership in complex or ambiguous technical landscapes.
Desirable skills/knowledge/experience: (As applicable)
- - Experience contributing to or leading data-platform re-architecture programs.
- - Exposure to Medallion/Lakehouse patterns or Databricks-style ecosystems.
- - Experience in regulated or high-assurance data environments.
- - Collaboration with modelling teams or environments where engineering and modelling intersect.