Job Description
Responsibilities:
Design, develop, maintain, and troubleshoot Jenkins CI/CD pipelines supporting Azure-based data engineering solutions.
Automate build, test, deployment, configuration, and release processes across development, test, QA, and production environments.
Support deployment/release processes for Azure data workloads, including Azure Data Factory, Azure Databricks, Azure Storage/Data Lake, and related services.
Partner with Data Engineers to incorporate data pipelines, notebooks, scripts, configuration, and infrastructure changes into repeatable deployment processes.
Manage Git branching, merging, pull requests, artifacts, and release processes.
Build pipeline-as-code and reusable automation to reduce manual deployment activities.
Troubleshoot failed builds, deployments, permissions, dependencies, credentials, and environment/configuration issues.
Support Azure identity, secrets, service connections, and secure access required by automated pipelines.
Implement deployment controls, approvals, logging, and traceability throughout the CI/CD lifecycle.
Support production deployments, operational troubleshooting, incident resolution, and root-cause analysis.
Maintain pipeline documentation, operational procedures, and support runbooks.
Participate actively in Agile/Scrum ceremonies, including sprint planning, stand-ups, backlog refinement, estimation, demos, retrospectives, and continuous improvement.
Required Skills
- Strong hands-on Jenkins experience
- CI/CD pipeline design, development, administration, and troubleshooting
- Microsoft Azure experience
- DevOps automation supporting data engineering workloads
- Git and enterprise source-control practices
- PowerShell, Bash, and/or Python scripting
- Azure Data Factory and/or Azure Databricks deployment experience
- Automated build, test, deployment, and release-management experience
- Agile/Scrum delivery experience
- Strong troubleshooting and root-cause-analysis skills
Preferred Skills
- Azure Databricks
- Azure Data Factory (ADF)
- Microsoft Fabric
- Azure Data Lake / Storage
- Azure DevOps
- Terraform / Infrastructure as Code
- Docker and Kubernetes
- YAML / JSON
- SQL and ETL/ELT understanding
- Monitoring/logging for CI/CD and Azure workloads
- Jira or Azure Boards experience