5

with the websockets and persistent TCP connections how are load balancers going to cope with the 64k port limit if they are handling a large farm of servers in the backend? need. someideas on setting up the infra for an app that can. potentially have 100k connections.

7
  • 1
    Short answer: The same way routers do it. Jun 30, 2012 at 5:01
  • What 64k socket limit?
    – womble
    Jun 30, 2012 at 9:38
  • @DavidSchwartz Bad answer. I am looking for details.
    – vivekv
    Jun 30, 2012 at 13:41
  • 1
    @womble Fixed the question I meant 64K ports
    – vivekv
    Jun 30, 2012 at 13:42
  • @vivekv: Don't think of a loadbalancer like a server. Think of it like a router. Jun 30, 2012 at 19:37

4 Answers 4

8

Your question appears to assume a SNAT (aka NAPT) translating load balancer. Here's some ideas about solving the 64k ephemeral ports problem. My experience is with F5 Networks' BIG-IP product (so the links are to their site), but the concepts are the same for other vendors:

  • Don't SNAT. If source ports are not translated, there will be no 64k limit. To turn off SNAT, you need to have the inside address of the load balancer set as the route (usually default route) on the inside servers.

  • Use a SNAT Pool. This makes a pool of internal IP addresses available to the load balancer to translate to. For example, two IP addresses in a SNAT pool will give you 128k ephemeral ports so 128k simultaneous TCP connections.

More advanced approaches:

  • Use "n-Path Routing" (that's F5's term, others may call it "Direct Server Return"). This doesn't translate the client address or port (or destination IP, for that matter!), so also makes the ephemeral port issue go away. The responses from the servers bypass the load-balancer. The way you achieve this is with loopback adapters hosting the same IP on all your servers, so that they'll accept the traffic.

I should point out that Websockets are a special challenge for traditional HTTP load balancers, as the connections live much longer - people do run into the ephemeral ports problem when they may never have before. In my view, the best solution is one that removes the SNAT requirement (first or third solutions above). Scaling is much improved, and the load on the load-balancer is reduced. The added complexity is worth it.

Here is a good article on the issue, from F5's Lori MacVittie: HTML5 Web Sockets Changes the Scalability Game

4

Keep in mind that a socket is a tuple of sec/dst address, src/dst ports and protocol and, as such, the number of permutations is a lot more than 64k. There are some situations where outbound connections on proxy servers might have issues based on particular implementations but port numbering hasn't been a big issue traditionally.

4
  • 1
    Yes I understand that a CONNECTION is represented by ((srcip, srcport), (destip, destport)) But on a server (assuming it is the dest) which essentially is the front end for a farm, the destport is limited to 64K max. This is traditional TCP restriction. Given that how are modern 100K + concurrent connections working?
    – vivekv
    Jun 30, 2012 at 13:47
  • 1
    @vivekv: By using a real load balancer, not a proxy.
    – womble
    Jul 1, 2012 at 0:06
  • 1
    @vivekv where are you getting this 64K max thing from? The only limit you have is for outgoing connections (due to available source ports). You can have 100k incoming connections on a single destination port, linux doesnt care one bit.
    – phemmer
    Jul 1, 2012 at 9:48
  • @Patrick see the link I have posted below to understand the problem better.
    – vivekv
    Jul 1, 2012 at 18:35
2

I found the most detailed answer to my question on StackOverflow.

1
  • 1
    It's a good answer, but very Linux-centric which is fine if you're building your own load balancer on Linux rather than using a hardware load balancer from a commercial vendor (more common in large deployments).
    – eey0re
    May 16, 2013 at 10:36
0

I know this question is old but I have some things to add in case someone else googles "WebSockets Load Balancer"...

WebSockets don't need load balancers. There, I said it. The reason is that browsers don't make outbound WebSocket connections until after the page has loaded.

So if the page is already done loading and you have access to execute JavaScript why on the world would you need a load balancer at that point? You wouldn't. You can do something simple like pick a ws:// or wss:// connection at random from an array or you can get fancy and make an AJAX call that returns a particular WebSocket server to connect to. You could even put the WebSocket URL into the page from the server-side code via a template.

Scaling up WebSocket applications is trivial... Just add more servers. Whenever you do that just add them to your outbound connection list. They can be located anywhere in the world--even at different domains/origins!

WebSockets don't have the cross-origin restrictions of regular HTTP(S) URLs either. Make a connection to wss://foo.com from http://bar.com. It'll work fine! Just about all of the traditional problems associated with scaling web applications have been solved with WebSockets. You can even deliver CSS, images, JavaScript, and anything else over the WebSocket once the connection is established (and cache these files locally in the browser now too!). Relegating load balancers and things like CDNs obsolete.

2
  • 2
    It's a fair point - WebSockets don't need load balancers, but if you want to be pedantic regular websites don't need load balancers either - clients could just be redirect to a random server from the pool. Back in reality, "load balancers" aren't deployed these days just for "load balancing" but for "Application Delivery" and are there to provide a control point for load management, SSL offload, compression, and various other optimisations that you can't do if connections are direct from client to server.
    – eey0re
    May 16, 2013 at 10:39
  • 1
    This post makes a ton of assumptions about how the application is built, and is basically inaccurate.
    – Brad
    Jan 12, 2015 at 20:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .