I'm currently building an infrastructure management tool that provisions bare metal and VM's etc. We have a worker VM that runs commands(via ansible) on the remote nodes over SSH.
One of the steps requires the rebooting of nodes to apply some configurations. The worker process has to run more commands on the nodes after the reboot is complete(must be done synchronously).
My question is, how can I check to see whether the reboot has completed?
I could add a sleep timer(to wait until the reboot completes), but I feel like that is a bad solution for a number of reasons.
Another option is to try to SSH to the remote node, from my worker process, every 5 seconds or so, if it fails, keep trying again until I get a successful connection.
Is there another way of doing this?