Job Description: Require 7+ years of experience.
Key Responsibilities
- Design, develop, & maintain data pipelines using Databricks, Apache Spark, & cloud technologies.
- Build & optimize ETL / ELT workflows for processing large-scale structured & unstructured data.
- Develop data ingestion frameworks from various sources including APIs, databases, files, & streaming platforms.
- Implement Delta Lake architecture & manage data lakes / data warehouses.
- Optimize Spark jobs for performance, scalability, & cost efficiency.
- Collaborate with Data Engineers, Data Scientists, Business Analysts, & stakeholders to deliver data solutions.
- Develop notebooks & workflows using Python, SQL, Scala, or PySpark.
- Ensure data quality, governance, security, & compliance standards are maintained.
- Monitor, troubleshoot, & resolve production data issues.
- Support CI / CD processes & automate deployment of data solutions.
- Create technical documentation & maintain best practices for data engineering.
Rate:
Not specified
Location:
Remote
IR35 Status:
Outside
Remote Status:
Remote
Industry:
Data & Analytics
Seniority Level:
Senior