11

I'm stuck trying to setup more than 65536 outgoing TCP connections from a Linux (RedHat5) box.

I have already configured both outgoing and accepting boxes to allow enough file descriptors.

I don't believe there is a problem on the accepting side - I have multiple target boxes, each with multiple IP addresses and I'm using multiple ports.

On the outgoing side I don't believe I'm hitting a limit per IP address - I'm using multiple IP addresses and ports (I'm opening connections from ports 30,000 - 60,000 for each of several IP addresses).

Is there some Linux kernel tunable parameter I'm missing? Or some fundamental limit in TCP?

The failure is that my app for opening the connections is hanging in the connect() call.

Thanks for any help NickB

3
  • What limit did you hit ? What was the error when you hit that limit ?
    – nos
    Jul 15, 2010 at 13:34
  • connect() hangs.
    – NickB
    Jul 15, 2010 at 13:36
  • 1
    What's in between your source and destiniation IPs? Are you possibly filling up some device's NAT table?
    – matt-dot-net
    Jul 15, 2010 at 14:03

3 Answers 3

10

Here is a blog where somebody got > 1,000,000 outbounds from a box.

  • Richard Jones, MetaBrew.com, 2008-11-04, A Million-user Comet Application with Mochiweb, Part 3, section Turning it up to 1 Million (Archived here.)

In Part 1, we set the range to "1024 65535" - meaning there are 65535-1024 = 64511 unprivileged ports available. Some of them will be used by other processes, but we'll never get over 64511 client connections, because we'll run out of ports.

...

So let's bring up 17 new IP addresses, with the intention of making 62,000 connections from each - giving us a total of 1,054,000 connections

1
  • Thanks! That works for me. Specifically, the sysctl seetings from metabrew.com/article/… allowed me to setup over 65,536 TCP connections.
    – NickB
    Jul 15, 2010 at 14:41
0

The TCP protocol only uses 16 bits for both destination and source port. There will be no way to have more than 65536 ports open at once - not even with Linux.

7
  • 3
    This is only true for one single IP address. If you use more than one, each IP address should be able to have 65536 open ports.
    – mtvec
    Jul 15, 2010 at 13:37
  • 1
    correct, a local port is required for each outgoing connection. IIRC, any ports already in use cannot be used for these connections. so, for example, if you're running telnet/ftp/http, ports numbered 21/23/80 are not available as local ports.
    – KevinDTimm
    Jul 15, 2010 at 13:39
  • 1
    You can also have multiple connections to the same port.
    – gtrak
    Jul 15, 2010 at 13:40
  • @Job - Yes, since TCP is stacked on top of IP - I am only talking about one IP address.
    – matt-dot-net
    Jul 15, 2010 at 13:58
  • 5
    Each connection is identified by a tuple consisting of the remote address and port and the local address and port. So > 65536 inbound connections from different remote hosts is possible. > 65536 outbound connections from a single interface on a host is not possible and you'd be limited to somewhat less than that anyway as there will always be some ports in use. Jul 15, 2010 at 14:33
0

In this context, a "single interface" means a single network care with a single IP address assigned to it. Each outbound TCP connection is bound to a separate port, so your example is not possible.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .