Job Description
Position:Databricks Platform Engineer
Location:Bloomington,IL ( 90% offsite, 10% onsite)
Duration:Long term
Must-Have Skills
- 6+ years of cloud platform or data infrastructure engineering experience, including 2+ years of production Databricks experience.
- Strong hands-on experience designing and administering Databricks workspaces on AWS.
- Deep AWS infrastructure experience with VPC, IAM roles and policies, cross-account access, SCPs, S3, KMS, Secrets Manager, and CloudTrail.
- Strong Terraform experience, specifically with the Databricks Terraform provider.
- Hands-on expertise with Unity Catalog administration, including metastore configuration, catalog and schema ownership, storage credentials, external locations, privilege management, and RBAC.
- Experience implementing column-level masking, row-level security, dynamic views, lineage, and audit logging.
- Strong knowledge of private networking, including AWS PrivateLink, VPC peering, transit gateways, security groups, NACLs, and no-public-IP configurations.
- Hands-on experience defining Databricks cluster policies, instance pools, job clusters, autoscaling, Photon, runtime standards, and cost controls.
- Experience with Databricks Workflows, system tables, CLI, SDK, Repos, and Git integration.
- Strong CI/CD experience using GitHub Actions, Azure DevOps, or equivalent for Databricks deployments.
- Experience implementing platform monitoring, cost attribution, and observability using Databricks system tables, Datadog, Grafana, or equivalent tools.
- Ability to create platform architecture documentation, operational runbooks, onboarding standards, and production support procedures.
Profiles should demonstrate all the following
- Production Databricks platform administration on AWS
- Unity Catalog implementation and administration
- Terraform-based Databricks provisioning
- AWS networking and cross-account IAM experience
- Databricks security, governance, cluster policies, and cost controls
- CI/CD automation for Databricks platform resources
Rate:
Not specified
Location:
Remote
IR35 Status:
Outside
Remote Status:
Remote
Industry:
IT
Seniority Level:
Senior