Azure Databricks Data Engineer (Remote) - W2
Posted Today by 1765353015
Negotiable
Outside
Remote
USA
Summary: The Azure Databricks Data Engineer role is a remote position focused on designing, building, and optimizing scalable data solutions on Azure, with a strong emphasis on PySpark, Airflow, SQL, and ETL pipelines. Candidates with prior healthcare industry experience are highly preferred. The position requires collaboration with cross-functional teams to ensure data quality and compliance with healthcare standards. The ideal candidate will possess strong analytical and troubleshooting skills to maintain high performance and reliability in data workflows.
Key Responsibilities:
- Design, develop, and maintain data pipelines using Azure Databricks and PySpark.
- Build scalable and efficient ETL workflows to support analytics, reporting, and operational needs.
- Orchestrate data workflows using Apache Airflow.
- Develop and optimize SQL queries for data transformation, validation, and integration.
- Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
- Ensure data quality, integrity, and compliance with healthcare data standards.
- Troubleshoot and resolve issues in data pipelines and ETL processes.
- Implement best practices for performance tuning, optimization, and cost management in Azure.
Key Skills:
- Strong hands-on experience with Azure Databricks.
- Proficiency in PySpark for large-scale data processing.
- Experience with Airflow for workflow orchestration.
- Strong SQL skills for data analysis and transformation.
- Proven experience building and maintaining ETL pipelines.
- Healthcare industry experience (required or highly preferred).
- Familiarity with Azure services such as ADLS, Azure Data Factory, Azure SQL, etc.
- Strong analytical, troubleshooting, and communication skills.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Job Description Azure Databricks Data Engineer (Remote)
Position Overview
We are seeking an experienced Azure Databricks Data Engineer with strong expertise in PySpark, Airflow, SQL, and ETL pipelines. This is a remote position and requires candidates who are s. Prior healthcare industry experience is highly preferred. The ideal candidate will design, build, and optimize scalable data solutions on Azure, ensuring high performance, reliability, and data quality.
Key Responsibilities
- Design, develop, and maintain data pipelines using Azure Databricks and PySpark.
- Build scalable and efficient ETL workflows to support analytics, reporting, and operational needs.
- Orchestrate data workflows using Apache Airflow.
- Develop and optimize SQL queries for data transformation, validation, and integration.
- Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
- Ensure data quality, integrity, and compliance with healthcare data standards.
- Troubleshoot and resolve issues in data pipelines and ETL processes.
- Implement best practices for performance tuning, optimization, and cost management in Azure.
Required Skills & Qualifications
- Strong hands-on experience with Azure Databricks.
- Proficiency in PySpark for large-scale data processing.
- Experience with Airflow for workflow orchestration.
- Strong SQL skills for data analysis and transformation.
- Proven experience building and maintaining ETL pipelines.
- Healthcare industry experience (required or highly preferred).
- Familiarity with Azure services such as ADLS, Azure Data Factory, Azure SQL, etc.
- Strong analytical, troubleshooting, and communication skills.
Preferred Qualifications
- Experience working with HIPAA-compliant data environments.
- Knowledge of data modeling and performance optimization techniques.
- Experience implementing CI/CD pipelines for data workloads.