W2 Role: Sr. Databricks Engineer (with Validation/Testing experience)

Summary: The role is for a Senior Databricks Engineer with a focus on validation and testing, requiring deep expertise in Databricks and experience with Medallion Architecture. The position is fully remote and is structured as a long-term contract. Candidates must have strong hands-on experience with data ingestion and testing methodologies within Databricks. Familiarity with CI/CD processes and additional testing tools is advantageous but not mandatory.

Key Responsibilities:

• Develop and implement solutions using Databricks with a focus on Medallion Architecture.
• Validate and test data as it transitions across different layers (Bronze, Silver, Gold).
• Ingest data from multiple source systems into data lakes.
• Utilize Databricks notebooks, Unity Catalog, Python, and PySpark effectively.
• Test and report on curated data layers, explaining QA/testing approaches clearly.
• Manage CI/CD processes in Databricks, including code promotion and developer workflows.
• Optionally utilize Playwright and TypeScript for testing.

Key Skills:

• Deep expertise in Databricks development.
• Strong experience with Databricks Medallion Architecture.
• Proficient in validating and testing data across layers.
• Experience with data ingestion from multiple source systems.
• Strong skills in Databricks notebooks, Unity Catalog, Python, and PySpark.
• Knowledge of CI/CD processes in Databricks.
• Familiarity with Playwright and TypeScript is a plus.
• Certifications in relevant areas are advantageous but not required.

Salary (Rate): undetermined

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT