1

I'm running http_load on OpenBSD 5.2, to test how good my nginx setup is, and I've noticed that cold runs are much faster than warm runs, and with every run the performance goes down very dramatically (e.g. from 3735 replies per second on a cold run, to 2288, 1804, 1553 on subsequent runs).

I've noticed with netstat -n | wc -l that there are several thousand connections after running http_load, most of which are in the (state) of TIME_WAIT.

It might seem like set timeout tcp.finwait 8 for pf.conf would reduce some timeout value from 45s to 8s, but it doesn't seem to affect these TIME_WAIT connections at all, which still stay in netstat -n for exactly 60s from the time they're created through http_load / nginx.

Is there a way to expire these TIME_WAIT connections much sooner than 60s?

Improve this question
6
  • 1
    You should not even consider this if you don't fully understand RFC 1337 -- TIME-WAIT Assassination Hazards in TCP. You should figure out why this is causing you a problem and fix that rather than trying to change the timing.
    – David Schwartz
    Feb 8, 2013 at 1:16
  • Yes, that RFC seems very relevant, but how exactly does it apply here? If http_load makes 1000 connections per second for 4 seconds, for a total of 3999 connections, then I end up with having 3831 connections in a TIME_WAIT (state) as per netstat (not sure why it's 3831, and not 3999), which obviously slows down various OS lookups and insertions, and also rather obviously slows down the speed at which new connections can be established.
    – cnst
    Feb 8, 2013 at 3:08
  • If former connections have to be kept in the TIME_WAIT (state) for 60s after FIN, doesn't it imply that a single IP-address is only good for at most 64K requests per minute, or, in other words, only 1000 requests per second? That seems kinda wrong, there's got to be a better way.
    – cnst
    Feb 8, 2013 at 3:16
  • You should find out why it slows those things down and fix that. There's no reason it should have those effects. If you need to establish more than 64K outbound TCP connections per minute, you're probably doing something very wrong. (FWIW, 30 seconds is generally regarded as safe.)
    – David Schwartz
    Feb 8, 2013 at 3:19
  • @DavidSchwartz, why do you say "outbound"? Don't you have to TIME_WAIT for any connection, outbound or inbound? The TIME_WAIT timeout is definitely 60s in OpenBSD, and I haven't yet found a way to change it, which will basically indeed limit the server to processing at most 1000 individual connections per second per IP address.
    – cnst
    Feb 8, 2013 at 4:25

1 Answer 1

Reset to default
1

You can play with these sysctl variables:

/etc/sysctl.conf

net.inet.tcp.keepinittime = 150
net.inet.tcp.keepidle = 14400
net.inet.tcp.keepintvl = 150
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .