Running multiple copies of the script - using cores or simply slowing the server down?

Question

I have a small compact PHP script that monitors a beanstalk message queue on a CentOS 5 server. When it gets a message, it starts to undertake a particular action and once complete, goes back to the queue and listens for more work.

Everything works great and no problems so far. A friend mentioned to me a few days ago during a random conversation that this would be an ideal script that could make use of multiple cores on the server. That rang a bell with me - good idea! If I need to have two processes running I thought I would need two servers, but possibly not.

So the question is - if I ran two, three, four ... x copies of the script on ONE single server, would it slow the execution of the other scripts down, or would they be unaffected? Given that the script is simply listening then actioning and going back to listen again, are there likely to be any conflicts? I appreciate the latter may be more of a programming question than a server one!

Answer 1

If this is running on apache, using mpm_prefork (because it's php), then the linux kernel will balance the children processes between multiple cores.

If you are running those on the command line, then they are not necessarily running on different cores, and making then run on different cores requires using taskset or cpuset to define processor affinity of each instance to a different core.

In theory, you would be ok by running a number of scripts equal or lesser than the number of cores on your server, but in practice you still need CPU to run the kernel and other software/daemons, so YMMV. All this is affected by type of application, connections, dependency to external software and network and so on, so you really have to test on your own infrastructure to be sure.

Answer 2

My understanding is thus: there is a message queue. A PHP script consumes messages from this queue. The action taken varies across disk access, network activity and CPU intensive tasks.

(what coredump said on managing to utilize multiple processes at all.)

Also, if there is latency in some tasks (e.g. talking across network links), you may even want more scripts than number of cores.

Ultimately, only measurement can tell. Build a test system and hammer it with a realistic message mix. It does not matter so much what sort of system you test on, as long as it actually has multiple cores. A virtual machine is fine; you will still get a decent feeling of what happens as you increase number of consumers.