I have a Linux server with an Erlang application which shows an odd limit of exactly 65536 max tcp incoming connections.
The Erlang application is written with the Cowboy framework.
I have tuned the kernel parameters in this way:
# Increase system file descriptor limit fs.file-max = 300000 # Discourage Linux from swapping idle processes to disk (default = 60) vm.swappiness = 10 # Increase Linux autotuning TCP buffer limits # Set max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE # Don't set tcp_mem itself! Let the kernel scale it based on RAM. net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.rmem_default = 16777216 net.core.wmem_default = 16777216 net.core.optmem_max = 40960 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 # Make room for more TIME_WAIT sockets due to more clients, # and allow them to be reused if we run out of sockets # Also increase the max packet backlog net.core.netdev_max_backlog = 50000 net.ipv4.tcp_max_syn_backlog = 30000 net.ipv4.tcp_max_tw_buckets = 2000000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 10 # Disable TCP slow start on idle connections net.ipv4.tcp_slow_start_after_idle = 0 # Disable source routing and redirects net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.accept_source_route = 0 # Log packets with impossible addresses for security net.ipv4.conf.all.log_martians = 1
* soft nofile 300000 * hard nofile 300000
I also checked the maximum number of Erlang processes, but it doesn't seem the limiting factor:
1> erlang:system_info(process_limit). 262144
The test connections are made by 4 client machines from 4 different IP addresses (so it isn't a problem with client limits) and after an initial authentication, the connections are left idle, so server CPU utilization is below 50% and memory utilization is also below 35%.
EDIT: I'm not sure if this is important: the Erlang server application is listening on port 8000 and I get it to listen on port 80 too with this iptables rule:
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000
What else should I look into to remove the 65536 connections limit?