Azure Databricks Architect

Azure Databricks Architect

Posted Today by TXP

Apply
£625 Per day
Inside
Remote
London
Apply
Azure Databricks Azure Data Factory Azure Data Lake Cloud Computing Cloud Technology Continuous Integration and Continuous Delivery Cost Management Databricks Data Governance Data Ingestion Data Security Management Microsoft Azure Project Cost Management Scalability Stakeholder Management

Summary: The Azure Databricks Architect will provide architectural leadership for the design and implementation of an Azure-based data and analytics platform in a public sector environment. This role emphasizes platform architecture, data governance, and compliance with organizational standards while collaborating with delivery teams. The architect will also focus on ensuring solutions are secure, scalable, and meet non-functional requirements. The position requires travel to London twice a month and operates inside IR35 regulations.

Key Responsibilities:

  • Define and maintain the end-to-end architecture for Azure Databricks-based data platforms.
  • Design Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), and associated Azure services.
  • Produce and maintain High-Level Designs (HLDs) and Low-Level Designs (LLDs).
  • Establish and govern data ingestion, transformation, and consumption patterns.
  • Design and oversee data security and access models, including role-based access control and data segregation.
  • Ensure alignment with public sector governance, security, and compliance requirements.
  • Address non-functional requirements including performance, resilience, scalability, and cost management.
  • Support architectural assurance activities and formal design review processes.
  • Define integration patterns with Azure services, including Azure Data Factory, event-based and batch ingestion services, and downstream analytics and reporting platforms.
  • Provide architectural guidance on CI/CD and Infrastructure-as-Code approaches.
  • Support platform optimisation and sustainable operating models.
  • Act as a design authority for Azure Databricks architectural decisions.
  • Provide guidance and oversight to delivery teams, including data engineers.
  • Support early-phase discovery and design for new data and analytics use cases.
  • Identify architectural risks, constraints, and dependencies and support mitigation planning.

Key Skills:

  • Demonstrable experience designing Azure Databricks architectures in enterprise environments.
  • Strong understanding of Databricks Lakehouse architecture, Delta Lake, and governance concepts.
  • Architectural-level knowledge of Spark-based platforms.
  • Experience designing secure, governed data platforms in regulated or public sector environments.
  • Proven ability to produce HLDs, LLDs, and architectural decision documentation.
  • Strong knowledge of Azure data and integration services.
  • Experience delivering data platforms within UK public sector or government organisations (desirable).
  • Familiarity with government cloud and security standards (desirable).
  • Experience supporting architectural assurance or design review boards (desirable).
  • Knowledge of enterprise analytics and reporting platforms (desirable).
  • Relevant Azure or Databricks certifications (desirable).

Salary (Rate): £625/day

City: London

Country: UK

Working Arrangements: remote

IR35 Status: inside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Azure Databricks Architect
6 months
Remote with 2 days travel to London per month
Active SC
Inside IR35

Role Summary
We require an Azure Databricks Architect to provide architectural leadership and assurance for the design, implementation, and ongoing evolution of an Azure-based data and analytics platform within a public sector environment. The role will focus on platform architecture, data patterns, governance, and non-functional requirements, working alongside delivery teams and stakeholders to ensure solutions are secure, scalable, and compliant with organisational and government standards.

Key Responsibilities

  • Define and maintain the end-to-end architecture for Azure Databricks-based data platforms.
  • Design Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), and associated Azure services.
  • Produce and maintain High-Level Designs (HLDs) and Low-Level Designs (LLDs).
  • Establish and govern data ingestion, transformation, and consumption patterns.
  • Design and oversee data security and access models, including role-based access control and data segregation.
  • Ensure alignment with public sector governance, security, and compliance requirements.
  • Address non-functional requirements including performance, resilience, scalability, and cost management.
  • Support architectural assurance activities and formal design review processes.
  • Define integration patterns with Azure services, including Azure Data Factory, event-based and batch ingestion services, and downstream analytics and reporting platforms.
  • Provide architectural guidance on CI/CD and Infrastructure-as-Code approaches.
  • Support platform optimisation and sustainable operating models.
  • Act as a design authority for Azure Databricks architectural decisions.
  • Provide guidance and oversight to delivery teams, including data engineers.
  • Support early-phase discovery and design for new data and analytics use cases.
  • Identify architectural risks, constraints, and dependencies and support mitigation planning.

Essential Skills and Experience
Demonstrable experience designing Azure Databricks architectures in enterprise environments. Strong understanding of Databricks Lakehouse architecture, Delta Lake, and governance concepts. Architectural-level knowledge of Spark-based platforms. Experience designing secure, governed data platforms in regulated or public sector environments. Proven ability to produce HLDs, LLDs, and architectural decision documentation. Strong knowledge of Azure data and integration services.

Desirable Skills and Experience
Experience delivering data platforms within UK public sector or government organisations. Familiarity with government cloud and security standards. Experience supporting architectural assurance or design review boards. Knowledge of enterprise analytics and reporting platforms. Relevant Azure or Databricks certifications.

Engagement Model
The role will operate in an architectural and advisory capacity with limited hands-on delivery. The architect will work across multiple teams and workstreams rather than being embedded in a single delivery team. The role will support both delivery and assurance activities throughout the lifecycle of the platform.

Apply
Similar Jobs
Loading data...