job summary
We are seeking an experienced Databricks Data Engineer to join our AI & Data team supporting Health & Life Sciences initiatives. In this remote role, you will design, build, and optimize scalable ETL/ELT pipelines and curated data products across large-volume structured and semi-structured datasets.
location: Telecommute
job type: Contract
salary: $48.66 - 50.00 per hour
work hours: 8am to 5pm
education: Bachelors
responsibilities
- Design, build and maintain scalable ETL/ELT pipelines using Databricks, PySpark and SQL.
- Develop data ingestion and transformation frameworks supporting large-volume structured and semi-structured datasets.
- Implement Delta Lake patterns, data-quality validations, exception handling and reconciliation processes.
- Optimize Spark workloads and SQL queries for performance and cost efficiency.
- Collaborate with analysts, architects and source-system teams to deliver reliable curated data products.
qualifications
REQUIRED QUALIFICATIONS
Minimum Bachelors of Engineering Degree.
5+ years of experience with Databricks, PySpark, SQL, Delta Lake, ETL/ELT, and large-scale data processing.
5+ years of experience with Unity Catalog, Azure/AWS, ADF, Databricks Workflows, CI/CD, and Performance Optimization.
4 - 6 years of overall Data Engineering experience.
Must be located in the United States (Remote).
PREFERRED QUALIFICATIONS
Prior industry experience working on Health Payer projects.
skills
AWS,Spark,AI,large-scale data processing,CI/CD,data ingestion,Delta Lake,ELT pipelines,Databricks,ETL,Data Engineering,Azure,Performance Optimization,PySpark,SQL,SQL queries,reconciliation processes,data products,Managed Services,exception handling,cost efficiency,Workflows