Required Qualifications
- Deep understanding of Databricks Lakeflow connect capabilities
- Extensive experience in addressing ingestion challenges from unsupported sources by Lakeflow connect. CDC handling and reconciliation techniques
- Expertise on Data quality libraries and experience putting together configuration based framework for plugin in DQ into ingestion pipelines.
- Deep understanding of AWS analytics and ingestion related services
- Extensive experience designing enterprise Databricks lakehouse architectures.
- Deep knowledge of Databricks, Spark, Delta Lake, Lakeflow, and Unity Catalog.
- Strong experience with medallion architecture and Bronze/Silver/Gold data patterns.
- Experience designing enterprise data governance, lineage, security, and access-control frameworks.
- Strong understanding of ABAC, RBAC, data classification, tagging, and least-privilege security.
- Experience designing production-grade data pipelines and reusable ingestion frameworks.
- Strong understanding of observability, data quality, CI/CD, deployment automation, and production support.
- Experience designing semantic layers and governed enterprise data products.
- Experience architecting AI/GenAI solutions that operate against governed enterprise data.
- Strong communication skills with the ability to work across engineering, security, architecture, and business teams.
Preferred Qualifications
- Databricks architecture or engineering certifications.
- Experience with Databricks Genie and AI/BI.
- Experience with MCP and agent-based architectures.
- Experience with enterprise GenAI governance, tracing, and evaluation.
- Experience integrating data across multiple enterprise business domains.
- Experience creating configuration-driven reusable data engineering frameworks.
IR35 Status:
Not specified