Summary: The Databricks Platform Engineer Administrator is responsible for providing daily support for the Databricks platform on AWS, ensuring data security, and optimizing performance. The role involves troubleshooting, managing AWS infrastructure, and collaborating with engineering and DevOps teams. Additionally, the position requires developing automation scripts and implementing CI/CD pipelines for efficient platform management. A strong background in Databricks and AWS services is essential for success in this role.

Key Responsibilities:

• Provide daily support for the Databricks platform running on AWS.
• Ensuring data security and access controls are setup and maintained.
• Monitor and troubleshoot clusters, workspaces, jobs, and other Databricks resources.
• Familiar with setting up and using medallion architecture, and the unity catalog.
• Handle incidents, requests, and changes in Databricks environments.
• Collaborate with engineering and DevOps teams to resolve complex platform issues.
• Familiarity with Databricks ML flows and monitoring.
• Analyse workloads and optimize Databricks clusters for performance and cost-efficiency.
• Recommend best practices for efficient Spark job execution and data processing.
• Manage and configure AWS services (eg, S3, EC2, IAM, Lambda) that integrate with Databricks.
• Ensure proper security configurations and compliance with AWS standards.
• Manage networking, security groups, and VPCs related to Databricks setup.
• Develop and maintain automation scripts for platform maintenance and monitoring.
• Work closely with DevOps teams to automate cluster management, scaling, and infrastructure provisioning.
• Implement CI/CD pipelines for Databricks deployments using tools like Jenkins, Git, or Terraform.

Key Skills:

• Bachelor's degree in computer science, Information Technology, or related field and 5+ years of experience supporting Databricks or similar data platforms.
• Strong expertise in Databricks on AWS: cluster management, job execution, Spark optimization.
• Hands-on experience with AWS services like EC2, S3, IAM, Lambda, RDS, VPC.
• Proficiency in Python, Scala, or SQL for data engineering tasks.
• Solid understanding of Apache Spark, ETL pipelines, and big data processing.
• Familiarity with DevOps practices, automation tools (Terraform, Ansible), and CI/CD pipelines.
• Experience with monitoring and logging tools (CloudWatch, Datadog, or similar).
• Strong problem-solving and troubleshooting skills.
• Excellent written and verbal communication skills.

Salary (Rate): undetermined

City: Coventry

Country: UK

Working Arrangements: undetermined

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT