So here is the scenario: Currently to cleanup space on Jenkins slave we have a job(a jenkins job itself) scheduled to run every day to do the cleanup. But many a times disk get full between scheduled runs then we have run it manually.
This is what I am looking for: Along with a schedule based cleanup I would like to have a demand based cleanup. Whenever a node disk get higher than say 70% that node should be taken offline, cleaned up and then put back online. All this should be automated. Is this possible?
I am a newbie to Jenkins so I can just speculate: Can we configure Jenkins to monitor its own slaves and trigger something(script etc) when a condition is met?
EDIT: I would not like to use extra softwares like nagios etc. to do the monitoring. What I am looking for is whether slaves themselves can say to master "we are unhealthy, save us!" or master can monitor slaves and do something when they are not healthy. Basically I want to know whether Jenkins has self-healing capability or I need to depend on a third party tool?