6

I'm configuring gunicorn (monitored by supervisord and behind an nginx frontend) and got a bit confused about the optimal number of processes to setup.

In the documentation it is clearly explained that:

workers = multiprocessing.cpu_count() * 2 + 1

My machine is a quad core so that should count for 9 workers.

But I want to run several applications, each one listening to a different port.

Should the count then be (truncated):

workers_per_application = int(workers / NUM_APPLICATIONS)

Or should each have the above number of workers?

I think this question actually applies not only to gunicorn but to every similar kind of listening server...

1
  • a 'gunicorn' tag does not yet exist on server fault?! How so? Please somebody with good rep. go create it!
    – Stefano
    Dec 14, 2011 at 16:32

1 Answer 1

5

Honestly, the workers_per_application is more of a performance tweak to ensure your application can consume 100% of the CPU at any time. It does not mean that it will. You can configure all of your applications to have 9 workers... as long as you keep in mind that there is a potential that one application could be working on something very difficult which would cause another to lag/fail-to-respond in time. The whole "cpu_count() * 2 + 1" thing is a suggestion, at-best... and you can add more to it... or less as you deem necessary. I am not sure if the cpu_count() returns the number of physical processors... or the number of CPU cores. A quad-core + hyper-threading might appear to be 8 cores which would translate as 17 processes... or it might only count as 1 ... translating as 3 processes. Fiddle with it & see what happens.

You may want to set a hard-number for that if you want 2 applications to run equally well without seeing much lag caused by the other.

2
  • Thanks! I guess that "ensure your application can consume 100% of the CPU at any time" is actually... BAD if I want to keep some spare resources for a database and some async jobs, isn't it? On the other side is "1" too little and it can result in often-blocked gunicorn because of one intensive request?
    – Stefano
    Dec 14, 2011 at 16:51
  • 1
    most of the time... applications like that won't ever really hit the CPU very hard. Tweaking it probably won't be necessary. In highly-scaled environments where you are really trying to squeeze every last drop of power you can out of your hardware... options like that can make all the difference.
    – TheCompWiz
    Dec 14, 2011 at 16:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .