Must-Have Experience
- 8+ years in Data Engineering, Data Architecture, or Data Platform roles
- 3+ years hands-on Databricks (Lakehouse, Delta Lake, Unity Catalog, PySpark)
- Medallion Architecture implementation (Bronze/Silver/Gold)
- Unity Catalog / data governance rollout experience
- Enterprise data governance — Role based access control, metadata management, data classification, lineage, access control
- Scalable ingestion, transformation, and provisioning pipelines
- Azure (ADF, DevOps, Synapse) and/or AWS (S3, Lambda, Glue, Redshift, MWAA)
- Python and PySpark development
Highly Desired
- Test Data Management (data masking, de-identification, synthetic data generation)
- AI-powered data discovery and classification
- Generative AI / Databricks Mosaic AI
- Data observability and monitoring frameworks
- Cross-team framework rollout and enablement
Nice to Have
- CI/CD and IaC (Terraform, Azure DevOps, GitHub Actions)
- Regulatory compliance (PII, GDPR, CCPA, HIPAA)
Key Responsibilities
- Roll out TDM framework across projects on Databricks
- Establish governance controls for sensitive data in lower environments
- Build scalable pipelines for test data generation, masking, and refresh
- Implement metadata management and lineage capabilities
- Design synthetic data strategies for non-production environments
- Enable self-service access to compliant test datasets
- Develop reusable, automated data provisioning and lifecycle frameworks
- Partner with QA, application teams, data governance, and security
Rate:
Not specified
Location:
Remote
IR35 Status:
Not specified
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Senior