I have provisioned a server with 8 cores and plan on deploying a network service. For spreading out request load I'd like to run 8 instances of my service. Nothing shocking here. I do not have access to a hardware load balancer. I should mention that I currently have allocated 5 public IP addresses (but I can get more).
Thus, I would like to hear your recommendations for structuring a software load balancing solution.
The obvious choices would be to either:
- use HAProxy; or
- pre-fork my application (like Facebook Tornado and Unicorn both do);
- insert your idea here.
My goals are to:
- spread request load between the service instances; and
- allow for rolling restarts of my service (code upgrades).
I should mention that this is not a HTTP-based service so NGiNX and the like is out.
I do not love HAProxy because of its memory requirements; it seems to require a read and write buffer per client connection. Thus, I would have buffers at the kernel level, in HAProxy, and in my application. This is getting silly! Perhaps I'm missing something in this regard though?