Negotiable
Undetermined
Remote
Remote
Summary: The Senior DataBricks Platform Engineer role focuses on leveraging Azure Cloud and Data Operations to support enterprise DataBricks environments. Candidates should possess extensive experience in cloud and platform engineering, with a strong emphasis on DataBricks and related technologies. The position requires collaboration with stakeholders and a solid understanding of DevOps practices. This role is fully remote, catering to experienced professionals in the data engineering domain.
Key Responsibilities:
- Support enterprise DataBricks environments and operations.
- Manage and optimize Delta Lake, Unity Catalog, and Spark workloads.
- Implement and maintain CI/CD pipelines and deployment automation.
- Collaborate with stakeholders to troubleshoot and provide operational support.
- Engage in enterprise data platform integrations and API support.
Key Skills:
- 6+ years of experience in cloud, platform engineering, or data engineering.
- 3+ years of hands-on experience with DataBricks.
- Strong knowledge of Azure cloud services and cloud-native concepts.
- Familiarity with Azure DevOps, GitHub workflows, and CI/CD pipelines.
- Experience with PySpark, Python, SQL, and Spark SQL.
- Understanding of AKS/Kubernetes, Terraform, and Infrastructure as Code.
- Strong troubleshooting and communication skills.
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Location: Remote
Domain Focus: DataBricks, Azure Cloud, Enterprise Data Platforms, Platform Engineering, DevOps Enablement
6+ years of experience within cloud, platform engineering, data engineering, or enterprise analytics ecosystems
3+ years of hands-on experience supporting enterprise DataBricks environments
Strong experience with:
Delta Lake
Unity Catalog
Spark workloads
cluster operations
DataBricks Workspaces
Experience with Azure cloud services and cloud-native platform concepts
Familiarity with:
Azure DevOps
GitHub workflows
CI/CD pipeline concepts
deployment automation
Experience supporting APIs, distributed application integrations, and enterprise data platforms
Strong troubleshooting and operational support capabilities
Strong communication and stakeholder collaboration skills
Preferred Qualifications
Experience with:
PySpark
Python
SQL
Spark SQL
Scala
Familiarity with:
AKS/Kubernetes
Terraform
Bicep
Infrastructure as Code practices
Experience with:
Azure Data Lake
Event Hub
streaming architectures
enterprise analytics environments
Familiarity with monitoring and observability platforms such as:
Azure Monitor
Datadog
Exposure to enterprise modernization and cloud migration initiatives