We're replacing a home-grown queueing system with SGE/OGE. The current work environment has engineers using their own local Linux workstation to submit jobs. So I'm wondering about adding many machines as submit hosts to an SGE/OGE cluster. In our case, there would probably be on the order of 100-200 submit hosts submitting jobs to a cluster that has approximately 30-50 execution nodes.
I think that's backward from what is "normal" where there is a single submit host (or maybe just a couple) which serves as the front-end to the cluster. Are there any down-sides to having lots of submit hosts (other than having to add them via qconf)?