JD:
Python Heavy, databricks, data eng
CI/CD tools (Jenkins, GitLab CI/CD)
Deployment automation
Docker (containerization)
Kubernetes (orchestration)
Data pipeline performance tuning
Data pipeline troubleshooting
Designing and building scalable, reliable, and efficient data pipelines
Debugging and troubleshooting data pipeline issues
Monitoring and optimizing data pipeline performance