Databricks Data Engineer

Introduction

The Databricks Data Engineer will be responsible for hands-on DLT/LDP, delta lake, unity catalog, DAB, spark advance performance tuning, and building modular engineering frameworks. This role will require experience with Foundry as a secondary skill, with an emphasis on specific requirements.

Responsibilities

• Hands-on experience with DLT/LDP, delta lake, unity catalog, DAB, and spark advance performance tuning
• Solid data engineering fundamentals and proven hands-on Databricks experience
• Performance tuning and optimization, including Spark internals, join strategies, and skew handling
• Efficient use of LDP APIs (CDF, CDC)
• Implementation of data quality frameworks, including expectations, quarantine logic, and alerting
• Compliance and governance tasks such as PII/PHI masking and access controls
• Orchestrating and stitching pipelines together

Requirements: Required Skills:

• Solid data engineering fundamentals
• Hands-on experience with Databricks
• Performance tuning & optimization (Spark internals, join strategies, skew handling)
• Efficient use of LDP APIs (CDF, CDC)
• Data quality frameworks (expectations, quarantine logic, alerting)
• Compliance & governance (PII/PHI masking, access controls)

Preferred Skills

Experience with Foundry

Unity catalog knowledge

Experience with orchestrating and stitching pipelines