I manage a Jenkins server (running 2.89.4) with about 40 nodes or so attached, each running either Centos7, OSX, or Windows (7, 8, 10). Recently, we suffered some power outages and many slaves lost their connection to the master. This required a lot of manual remoting and ssh'ing to reconnect them all. Since, I have made sure each slave process is running a server, however I'd like an additional safety net of being able to automatically restart every vm and/or box these slaves are running on. The problem is, since there are several different types of operating systems, there isn't a simple way to do it (that I know of yet).
I've tried using a Groovy script, but that can mostly just be used for restarting the slave service, not the machine itself. This was my reference: https://wiki.jenkins.io/display/JENKINS/Monitor+and+Restart+Offline+Slaves
I've also looked into an Ansible approach, but many machines are unreachable, and Ansible requires different types of scripts for Windows vs. Unix.
The best approach I've seen so far is here: https://stackoverflow.com/questions/29165080/restart-jenkins-slave-from-master
But it doesn't explain how to run this on every single slave, it just looks like it executes on one slave. Has anyone else done anything like this before? I'd love to create a single job, or single script than can be executed and it restarts every single node (using a command like
shutdown -r -f) connected to Jenkins regardless of platform.