Linux - Running The Same Command on Many Machines at Once
Here's the situation:
- We have a lab that consists of fifteen quad-core machines, each running Ubuntu Linux.
- There is a video encoding software I need to be running, but one job (i.e. one video with one configuration) takes a really long time (couple of hours).
- There are about as many configurations as machines (about 15) and around 20 videos.
So I thought about having the videos accessible through a centralized storage, but let each machine run the encoding process.
In its most basic form, the command used is something like this
./encode -d default.conf -f local.conf -i inputFile.yuv
Now, the question is: Is there any software that I could use to easily deploy those tasks on the lab machines? I was thinking about:
- Having one master that dispatches jobs, e.g. "Tell machine 1 to run
/home/user/encode -i input1.yuv, then
/home/user/encode -i input2.yuv, et cetera"
- Being able to see which node is currently working on which task and for how long
- Being able to stop a task or retry upon failure
I am not limited to CLI, could also be a GUI application. Any ideas?