Senior Databricks Developer
Posted Today by 1756883710
Negotiable
Outside
Remote
USA
Summary: The Senior Databricks Developer role is a part-time position focused on architecting and delivering a proof-of-concept (POC) that evolves into a comprehensive data platform for retail clients. This hands-on role requires extensive technical expertise in Databricks, Data Warehousing, and ETL development. The candidate will be responsible for building scalable data pipelines and ensuring governance and cost efficiency from the outset. Collaboration with internal stakeholders to define data requirements is also a key aspect of the role.
Key Responsibilities:
- Lead design and delivery of POC pipelines and data models on Databricks.
- Build scalable data pipelines (ETL/ELT) to integrate data from multiple retail sources (transactional DBs, APIs, 3rd-party datasets).
- Partner with internal stakeholders to define data requirements for the retail-focused platform.
- Productionize POC into a secure, cost-optimized, and performant Databricks Lakehouse platform.
- Implement governance (Unity Catalog), data modeling, and performance tuning practices.
- Enable analytics and reporting use cases for retail clients.
- Document architecture and provide knowledge transfer to internal teams.
Key Skills:
- 7+ years in Data Engineering with 3+ years Databricks (PySpark, Spark SQL, Delta Lake).
- Strong background in Data Warehousing or ETL pipeline development.
- Experience building POCs and scaling them to production platforms.
- Hands-on with Delta Lake, Unity Catalog, Databricks Workflows/Airflow.
- Strong SQL and data modeling skills for retail/transactional data.
- Cloud experience (Azure or AWS).
- Excellent communication skills and ability to work with product/engineering stakeholders.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Job Title: Senior Databricks Developer (Part-Time)
Location: 100% Remote
Engagement: Part-time (20 hrs/week, initial 2 months, with potential extension/full-time)
We are seeking a Senior Databricks Developer to help architect and deliver a proof-of-concept (POC) and then evolve it into a full-fledged data platform to serve retail clients. This is a hands-on role requiring deep technical expertise in Databricks, Data Warehousing, and ETL development.
The candidate will play a key role in shaping the data backbone of the new platform-building scalable pipelines, enabling analytics, and ensuring governance and cost efficiency from day one.
Responsibilities-
Lead design and delivery of POC pipelines and data models on Databricks.
-
Build scalable data pipelines (ETL/ELT) to integrate data from multiple retail sources (transactional DBs, APIs, 3rd-party datasets).
-
Partner with internal stakeholders to define data requirements for the retail-focused platform.
-
Productionize POC into a secure, cost-optimized, and performant Databricks Lakehouse platform.
-
Implement governance (Unity Catalog), data modeling, and performance tuning practices.
-
Enable analytics and reporting use cases for retail clients.
-
Document architecture and provide knowledge transfer to internal teams.
-
7+ years in Data Engineering with 3+ years Databricks (PySpark, Spark SQL, Delta Lake).
-
Strong background in Data Warehousing or ETL pipeline development.
-
Experience building POCs and scaling them to production platforms.
-
Hands-on with Delta Lake, Unity Catalog, Databricks Workflows/Airflow.
-
Strong SQL and data modeling skills for retail/transactional data.
-
Cloud experience (Azure or AWS).
-
Excellent communication skills and ability to work with product/engineering stakeholders.
-
Knowledge of MLflow, Databricks SQL Warehouses, DLT.
-
Experience with retail data (POS, inventory, customer behavior analytics).