What You ll Do
- Design, build, and maintain automation to patch production SQL Server Always On Availability Group clusters with no unplanned interruption.
- Automate discovery of availability group topology (replicas, roles, synchronization health, routing, current state).
- Automate pre-patch steps, including:
- Removing secondary replicas from read-only routing
- Placing nodes into monitoring maintenance mode
- Stopping services on DR availability group instances in a controlled manner
- Orchestrate safe patch sequencing across secondary, DR, and primary replicas, including failover/failback activities.
- Keep availability groups online throughout the patch cycle and validate health at each stage.
- Automate suppression and restoration of monitoring alerts (SQL Sentry / SentryOne; equivalents considered).
- Integrate patch execution with JAMS scheduled jobs and maintenance-mode workflows.
- Build reusable, parameterized Ansible Automation Platform playbooks/roles/job templates.
- Replace manual scripts and legacy SCORCH runbooks with standardized automation.
- Integrate automation with ServiceNow for change workflows:
- Change creation
- Approval gating
- Execution status updates
- Automated closure
- Implement verification steps, halt conditions, rollback logic, and restartability at each stage.
- Produce post-patch evidence (replica health, sync status, patch level, audit/vulnerability evidence).
- Support emergency and out-of-cycle database patching for high-severity vulnerabilities and CISA KEV items.
- Partner with DBA, Security, Infrastructure Ops, and App teams on maintenance windows and cutover planning.
- Support remediation of database-tier vulnerabilities identified through Tenable (or similar).
- Maintain automation in source control (Azure DevOps) and participate in code reviews and documentation (runbooks/SOPs).
Required Skills & Experience
- Strong hands-on SQL Server Always On Availability Groups expertise, including:
- Replica roles and synchronization modes
- Read-only routing, listeners
- Failover/failback
- Quorum behavior
- 6+ years enterprise Windows Server infrastructure engineering.
- 4+ years SQL Server Always On Availability Groups in production HA environments.
- 3+ years infrastructure automation using Ansible and/or PowerShell.
- Demonstrated success automating patching/maintenance for clustered or HA database environments.
- Experience operating in formal change-management and documentation standards.
- Regulated environment experience (HIPAA, PCI, SOX) preferred.
- Strong Windows Server Failover Clustering (WSFC) administration/troubleshooting.
- Experience patching and maintaining highly available SQL Server infrastructure in production.
- Advanced PowerShell scripting.
- Experience with SQL Server PowerShell module, dbatools, or similar tooling.
- Strong T-SQL skills for querying AG state and DMVs.
- Strong Ansible experience; Ansible Automation Platform (AAP) preferred.
- Experience building Ansible playbooks, roles, inventories, credentials, and job templates.
- Enterprise Windows patch management experience (MECM/SCCM, rings, maintenance windows).
- ServiceNow API integration experience (change records, approvals, automation status).
- Experience with enterprise scheduling tools (JAMS, Control-M, or similar).
- Monitoring platform integration experience (SQL Sentry/SentryOne, SCOM, or equivalent).
- Git-based workflows; Azure DevOps Repos and Pipelines experience.
- Ability to build idempotent, restartable automation with validation and rollback controls.
- Solid understanding of vulnerability remediation and severity-based SLAs.
- Strong troubleshooting, risk management, and communication skills.
- Bachelor s degree in Computer Science, Engineering, or related field (or equivalent experience).
- Microsoft certifications (e.g., Azure Database Administrator Associate; Windows Server-related)
- Red Hat Certified Specialist in Ansible Automation
- ServiceNow certifications
- ITIL Foundation