I have an architecture question. In a clustered web app environment, I can think of three ways to deal with background jobs:
- have a dedicated machine run all the jobs, thus freeing the web servers from having to do so
- have each web server also run background jobs, using a mechanism to make sure no two machines kick off the same job
- have one of the web servers double up as jobs-runner
What's the preferred approach?