I'm configuring gunicorn (monitored by supervisord and behind an nginx frontend) and got a bit confused about the optimal number of processes to setup.

In the documentation it is clearly explained that:

workers = multiprocessing.cpu_count() * 2 + 1

My machine is a quad core so that should count for 9 workers.

But I want to run several applications, each one listening to a different port.

Should the count then be (truncated):

workers_per_application = int(workers / NUM_APPLICATIONS)

Or should each have the above number of workers?

I think this question actually applies not only to gunicorn but to every similar kind of listening server...

  • a 'gunicorn' tag does not yet exist on server fault?! How so? Please somebody with good rep. go create it! – Stefano Dec 14 '11 at 16:32

Honestly, the workers_per_application is more of a performance tweak to ensure your application can consume 100% of the CPU at any time. It does not mean that it will. You can configure all of your applications to have 9 workers... as long as you keep in mind that there is a potential that one application could be working on something very difficult which would cause another to lag/fail-to-respond in time. The whole "cpu_count() * 2 + 1" thing is a suggestion, at-best... and you can add more to it... or less as you deem necessary. I am not sure if the cpu_count() returns the number of physical processors... or the number of CPU cores. A quad-core + hyper-threading might appear to be 8 cores which would translate as 17 processes... or it might only count as 1 ... translating as 3 processes. Fiddle with it & see what happens.

You may want to set a hard-number for that if you want 2 applications to run equally well without seeing much lag caused by the other.

  • Thanks! I guess that "ensure your application can consume 100% of the CPU at any time" is actually... BAD if I want to keep some spare resources for a database and some async jobs, isn't it? On the other side is "1" too little and it can result in often-blocked gunicorn because of one intensive request? – Stefano Dec 14 '11 at 16:51
  • 1
    most of the time... applications like that won't ever really hit the CPU very hard. Tweaking it probably won't be necessary. In highly-scaled environments where you are really trying to squeeze every last drop of power you can out of your hardware... options like that can make all the difference. – TheCompWiz Dec 14 '11 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.