Senior Data Architect

We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you.

Role Title: Senior Data Architect
Location: London (Hybrid -2/3 days per week)
Duration: 31/12/2026
Pay rate: £492 per day all inc. (PAYE through umbrella)

Job Description

Senior Solution Architect - Big Data & Streaming Platforms

Role Purpose

• We are seeking a senior solution architect with deep experience designing and delivering enterprise-scale big data, streaming, and Real Time data integration solutions.
• The role is focused on solution architecture across Cloudera-based big data platforms, Kafka-based streaming applications, and Real Time data integration/analytics solutions, including cloud-hosted data platforms such as Cloudera on AWS or equivalent AWS-native services.
• This is not a pure data modelling, reporting, analytics, or governance role. The successful candidate must have a strong architecture background and be comfortable operating across both solution design and hands-on technical delivery.

Key Responsibilities

• Own the end-to-end solution architecture for large-scale distributed data, streaming, and data integration solutions.
• Design solutions across Cloudera CDH/CDP, data lakes, operational data stores, event streaming platforms, and Real Time data pipelines.
• Architect and document solutions using technologies such as Hive, Impala, HBase, Spark, NiFi, Flink, Kafka, and equivalent AWS services.
• Design Kafka-based streaming applications, including event ingestion, topic design, consumer patterns, error handling, replay, scalability, and resilience.
• Define solution patterns for Spring Kafka consumers, ingestion into HBase, and consumption through Spring Boot/API-based services.
• Provide architecture leadership across data ingestion, streaming integration, transformation, storage, APIs, and downstream consumption.
• Work with engineering teams to ensure solutions are deliverable, scalable, secure, supportable, and aligned to enterprise architecture principles.
• Review technical designs produced by engineering teams and ensure alignment with the approved solution architecture.
• Identify technical risks, dependencies, assumptions, non-functional requirements, and design trade-offs.
• Support delivery planning by helping break solutions into iterative delivery increments and sprint-level outcomes.
• Engage with business stakeholders, programme teams, enterprise architects, engineering leads, and governance forums.
• Build, represent and own solution designs(Architecture Summaries and High level designs) to architecture review boards and senior technical stakeholders where required.
• Develop and promote architectural patterns and best practices and engineering standards across big data, streaming, integration, APIs, and cloud-based data platforms.

Essential Experience

• Candidates must demonstrate the following:
• 10+ years' overall IT experience, with significant experience in data platform, integration, or distributed systems delivery.
• 5+ years' experience in solution architecture, technical architecture, or lead design roles.
• 2+ years' experience architecting streaming or Real Time data applications, preferably using Kafka or equivalent event streaming technologies.
• Proven experience designing solutions on Cloudera CDH/CDP or equivalent big data platforms.

Strong demonstrable experience with some of the distributed data technologies (if not all) such as:

• HBase
• Spark
• NiFi
• Kafka
• Flink
• Hive
• Impala
• Experience designing and delivering large-scale data ingestion, transformation, storage, and consumption patterns.
• Experience with Kafka consumer architecture, including consumer groups, offsets, retries, dead-letter handling, ordering, partitioning, and scalability.
• Experience with API-based data consumption, ideally using Spring Boot/Java-based services.
• Strong understanding of batch, micro-batch, and Real Time streaming architectures.
• Experience working in complex enterprise environments with multiple systems, vendors, platforms, and governance processes.
• Strong understanding of non-functional requirements including performance, scalability, resilience, security, observability, and operational support.

Required Technical Skills

The successful candidate should have hands-on exposure to several of the following:

• Cloudera CDH/CDP, ideally on AWS or hybrid cloud environments
• Apache Kafka or AWS equivalents such as MSK, Kinesis, EventBridge, or similar
• Spark/Spark Streaming
• HBase/Cassandra/NoSQL data stores
• Hive/Impala
• Apache NiFi/ingestion frameworks
• Apache Airflow Orchestrations
• Apache Flink or equivalent stream processing technologies
• Java/Spring Boot/Spring Kafka
• Python and PySpark
• REST APIs and service-based data consumption
• Change Data Capture and event-driven integration patterns
• Data lake, ODS, and distributed storage design
• Cloud data platform design, preferably on AWS

Desirable Experience

• Financial services or banking experience.
• Experience with regulated, secure, enterprise-scale data platforms.
• Experience presenting to architecture governance boards.
• Exposure to enterprise data governance, lineage, metadata, and data quality tooling.
• Experience with DevOps, CI/CD, automated testing, and modern engineering delivery practices.
• Experience with cloud-native data services on AWS.

Candidate Profile

• This role is suited to a hands-on Senior Solution Architect who can design and guide the delivery of complex data and streaming solutions.

The ideal candidate will:

• Be an architect first, with strong hands-on engineering awareness.
• Have designed real production solutions using Kafka, Cloudera, Spark, HBase, APIs, or equivalent technologies.
• Understand both architecture-level trade-offs and engineering-level implementation constraints.
• Be comfortable challenging, reviewing, and improving technical designs.
• Be able to work with senior stakeholders while also engaging deeply with engineering teams.
• Have practical experience delivering scalable, resilient, secure, Real Time or near-Real Time data platforms.

Important ClarificatioN

• This is not a role for candidates whose experience is primarily in:
• Data analysis
• Data modelling only
• BI reporting
• Dashboard development
• SQL reporting
• Data governance only
• Business analysis
• Power BI/Tableau reporting
• Conceptual data architecture without implementation experience
• Candidates must have demonstrable experience in solution architecture for distributed big data platforms, streaming systems, event-driven integration, or Real Time data applications.

If you are interested in this position and would like to learn more, please send through your CV and we will get in touch with you as soon as possible. Please note, candidates are often Shortlisted within 48 hours.