tcp client which makes many outgoing connections

Question

My TCP client need to connect to thousands servers (in local network) all at once. Communication is simple. 10 bytes string TCP request, 20 bytes response.

I have gigabit network.

when client connects servers one by one no issues. But when all at once I have success with 1000 servers and others failing with errors: normally no route to host.

My clients running on brandnew NUC with i7. I was tweaking TCP stack :

sysctl -w fs.file-max=100000
mtu 500, 3000, 9000
ulimit -n 32000 2000 3000 5000 

But no success. Best I managed to get is 1800 connections Do u know how to overcome this issue?

Answer 1

I have managed to fix that issue so let me share how: with dmesg i find out that:

> [145307.501197] neighbour: arp_cache: neighbor table overflow!
> [145307.567830] neighbour: arp_cache: neighbor table overflow!
> [145308.354244] neighbour: arp_cache: neighbor table overflow!
> [145308.479650] neighbour: arp_cache: neighbor table overflow!

so solution was to increase ARP table:

sysctl -w net.ipv4.neigh.default.gc_thresh1=16000
sysctl -w net.ipv4.neigh.default.gc_thresh2=20000
sysctl -w net.ipv4.neigh.default.gc_thresh3=24000