Wouldn't it be better to determine why the jobs are failing and address that? – joeqwertyJan 31 at 17:35
Oh, certainly. :) But on our Linux clusters we typically put "just-in-case" sanity checks in the prologue and epilogue slots of Torque or Grid Engine, to catch things before jobs fail.
I'd like to find a way to put something comparable in place for Windows HPC. Especially since we have seen failure conditions in the past which don't get nodes flagged in WinHPC by default... for example, if a compute node loses contact with an SMB share. It generates an application error, but never offlines the nodes to prevent other jobs from hitting it. Putting that check in a sanity script would be useful. – ajdeconJan 31 at 21:12
This is a collaboratively edited question and answer site for system administrators and desktop support professionals. It's 100% free, no registration required.
Got a question about the site itself? meta is the place to talk about things like what questions are appropriate, what tags we should use, etc.
