Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I know just enough networking to be dangerous. The nitty gritty low level details of NAT are not something I am particularly knowledgeable about.

I accidentally found myself in a discussion earlier today about placing a bunch of our nodes behind a NAT Gateway. (1 public IP address and X private LAN addresses). I called up the 16 bit limit to source and destination port fields in the TCP protocol, (http://www.ietf.org/rfc/rfc793.txt - page 15) and mentioned that it would limit us to some 65,000 connections (65536). -- I am not so confident about that answer anymore. Can you help me with some details?

I understand that an incoming port (server port) on our side can accept as many connections as there are sourceIP x SourcePort combinations. Let's discount those for the time being and focus on connections originating in the LAN, traveling through the NAT Gateway, and ending on a random host at a random port.

On a normal [Linux] system, outgoing connections I believe are limited to 1 per port per Source IP. If we pretend that we live in a simple world where each system only has 1 IP address, then a 'normal system' would be limited to an absolute maximum of 65536 connections.

1) In TCP is a single source IP limited to 65536 MAX theoretical outgoing connections?

2) Or is the limit actually 65536 connections for each Remote Host?

2) [Written another way]: Can the same source port be used for a different remoteHostIP:RemotePort combination?

For example: (Is the following OK?)

Source IP   |Source Port |Remote IP|Remote Port   
192.168.0.20:36500   -->    8.8.8.8:23
192.168.0.20:36500   -->    8.8.4.4:23

3) Are the answers to questions 1 and 2 different for a ...'not normal system' [Cisco router acting as a NAT Gateway]?

Ex: A specialized networking device that has one public facing IP and up to ~65,000 Lan IPs [or more] behind it? Is there magic at place or is the answer to question 2 just always: yes? (or no)

4) The above questions all assume a stateful TCP connection. Is the story any different with a stateless conection like UDP?

And Ultimately:

5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?

Thank you! :)


For purposes of this question, we are behind very BEEFY AND BRAND NEW Cisco Nexus gear (7000 series I think). It may be better to ignore memory/etc limitations unless they can be specifically quantified.

share|improve this question
    
serverfault.com/questions/533611/… - Deals with incoming connections [A server serving x unique clients on its own server port]... I am asking specifically about outgoing connections through a NAT Gateway. socpuppet.blogspot.com/2013/03/… -- That 'blog post' suggests that port exhaustion is a real problem when there are many hosts behind a gateway and suggests "NAT Groups" and "NAT POOLS" -- breaking the LAN up and placing behind separate public IPs as a solution? -researching –  lVlint67 Sep 26 '13 at 0:04

3 Answers 3

up vote 3 down vote accepted

Correct me if I'm wrong but this is the way I understand it. The limits are per client / server / port. So in light of that.

1) In TCP is a single source IP limited to 65536 MAX theoretical outgoing connections?

No. I believe it's limited to 65536 theoretical max to the same destination IP.

Windows workstations (non server versions) have limits imposed which make this number much less. Linux has resource limits, but they generally aren't hit by the average user and you can easily tweak them.

You'll probably hit other resource limits as you start increasing the number anywhere near 64K.

Consumer grade routers might have much lower limits due to the limited resources.

2) Or is the limit actually 65536 connections for each Remote Host?

Yes

3) Are the answers to questions 1 and 2 different for a ...'not normal system' [Cisco router acting as a NAT Gateway]?

No

4) The above questions all assume a stateful TCP connection. Is the story any different with a stateless conection like UDP?

UDP is connectionless. So this isn't really relevant to UDP.

5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?

No.


In the context of stateful firewalls that track connections and provide other tracking features, yes these modules themselves may have limits. The op has not said anything about which firewall/NAT router is being used so we can't even speculate as to what limitations it might impose at the moment.

share|improve this answer
2  
UDP is affected by the same limits. When a UDP packet leaves a NAT firewall it has to hold that port and remote ip address open for a certain amount of time for return packets to make it through. –  longneck Sep 26 '13 at 2:05
    
'Other resource limits' are not particularly interesting in our discussion. I suspect you may be correct in some (or all) of your answers. Can you provide any supporting documentation? –  lVlint67 Sep 26 '13 at 2:25
1  
File handles, memory, cpu overhead. You can have lots and lots of active, open connections but there is overhead and what overhead varies between operating systems. –  Matt Sep 26 '13 at 3:34
    
@longneck - you're right in the context of NAT of course. Especially for stateful firewalls. –  Matt Sep 26 '13 at 3:36
1  
As per the Socket Reuse comments on user8162's answer, It is conceivable to build a NAT platform that uses the full 5 part tuple (protocol [udp/tcp], source ip, destination ip, source port, destination port) as a unique key for each outgoing socket. Host1:port1 -> host2:port2 and Host1:port1 -> host3:port2 are conceivable layouts that could in theory point to separate sockets. With that in mind, until I see something in the TCP spec or other official document that prevents this functionality, I will have to accept Matt's answer as correct. (taking into account the udp considerations). –  lVlint67 Sep 26 '13 at 5:43

Because there are many approaches to NAT design, the answer of course is "it depends." Geoff Huston wrote an excellent overview of various NAT designs.

In my experience many small-office/home-office routers will not handle > 64k addresses to the same address, but not just because of port exhaustion. Many have limited memory and will run out of RAM for a NAT table well before hitting the 64k limit.

share|improve this answer
    
That's a very lengthy article. Can you summarize the parts which are relevant for this question? –  Michael Hampton Sep 26 '13 at 1:23
    
First: The article is indeed very good (if lengthy). I can't find the specifc to my answer in my reading of it, but i suspect the answer may be hiding behind the mention of binding and mapping methods... and more specifically potentially the "endpoint independent" methods. Secondly, we have the luxury of not being limited by consumer grade hardware. We have pretty beefy Cisco Nexus 7000 series equipment at the edge. While I can't find any datasheets that give a MAX Connection limit... I assume it is well above the 65536 limit. –  lVlint67 Sep 26 '13 at 1:47
1  
    
Sorry if I was unclear -- the answer depends on what kind of architecture the NAT box uses. If it's just a Linux host (or one of many commercial products based on the Linux IP stack), then yes, 64k connections per Internet-facing IP address is the theoretical limit. Again, though, there are memory constraints with NAT table sizes that may mean the practical limit is much smaller than 64k. –  user8162 Sep 26 '13 at 2:31
1  
You asked about "other resource limits" being an issue with NAT. Besides my own experience of NAT boxes (both enterprise and soho grade) handling < 64k connections, there's also Cisco documentation suggesting that memory and CPU constraints come into play. –  user8162 Sep 26 '13 at 2:53

5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?

No, because one port NAT IP can be used for multiple connections:

cat /proc/net/ip_conntrack | grep 51380
tcp      6 191 ESTABLISHED src=10.1.8.5 dst=17.133.254.23 sport=51380 dport=5223 src=17.133.254.23 dst=my.nat.pub.ip sport=5223 dport=51380 [ASSURED] mark=0 use=2
tcp      6 24 CLOSE_WAIT src=10.1.26.1 dst=80.68.255.71 sport=51380 dport=80 src=80.68.255.71 dst=my.nat.pub.ip sport=80 dport=51380 [ASSURED] mark=0 use=2
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.