Job Summary
We are seeking an experienced Databricks Consultant to support a long-term client project in Washington State.
The ideal consultant will have strong hands-on experience with Databricks, SQL, Python, PySpark, and cloud-based data engineering solutions, with excellent communication and problem-solving skills and the ability to work independently in a contract-based engagement.
Scope of Work / Deliverables
Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks.
Develop data processing solutions using Python, PySpark, and Spark SQL.
Build and manage workflows using Databricks Workflows.
Work with Delta Lake for data storage, optimization, and transaction management.
Develop data ingestion pipelines from structured and unstructured data sources.
Implement data transformation, cleansing, and validation processes.
Optimize Spark jobs through partitioning, caching, and cluster tuning.
Integrate Databricks with cloud platforms such as Azure, AWS, or Google Cloud Platform.
Implement data security, governance, and access controls.
Collaborate with data architects, analysts, and business stakeholders.
Troubleshoot pipeline failures and ensure data quality.
Required Technical Skills
Strong hands-on experience with Databricks, Apache Spark, and PySpark.
Strong programming skills in Python.
Advanced SQL and database concepts.
Experience with Delta Lake and Delta Live Tables / Lakeflow Spark Declarative Pipelines.
Experience developing ETL/ELT data pipelines.
Knowledge of data warehousing and data modeling.
Experience with Azure Data Lake, AWS S3, or Google Cloud Storage.
Experience with Git, CI/CD, and deployment processes.
Understanding of data governance and security.
Preferred Skills
Experience with Azure Databricks.
Experience with Unity Catalog.
Knowledge of Kafka or real-time data streaming.
Familiarity with Terraform and infrastructure automation.
Knowledge of Power BI or other reporting tools.
Required Experience
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
6+ years of overall IT experience.
Minimum 3+ years of hands-on Databricks experience.
Strong analytical and problem-solving skills.
Excellent verbal and written communication skills.
Ability to work independently and manage project responsibilities.
Contract Details
1099 Independent Contractor engagement only.
Hourly compensation based on mutually agreed contract terms, invoiced by the contractor.
No W-2 employment benefits, paid leave, or withholding are provided.
Contractor is responsible for own taxes, insurance, equipment, and expenses.
Contractor must be an independently established business, maintain own books and records, file own schedule of expenses, and be free to perform similar services for other clients.
Contractor controls the method and means of performing the work, subject only to the deliverables and timelines in the SOW.
Interested candidates
Please share your updated resume, current location, work authorization, and expected hourly rate.