2

I'm trying to figure out how to consistently turn off keepalive across various client machines which are issuing HTTP requests via curl.

This is my target server:

- Ubuntu 18.04.2 LTS
- 4.15.0-47-generic
- HA-Proxy version 1.8.19-1ppa1~bionic 2019/02/12

This is client 1 where from I'm issuing curl (vanilla installation):

- Ubuntu 16.04.3 LTS
- 4.4.0-62-generic
- curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3

This is client 2 where from I'm issuing curl (vanilla installation):

- Ubuntu 18.04 LTS
- 4.15.0-20-generic
- curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3

To turn off keepalive I've tried using -H "Connection: close", --no-keepalive and --keepalive-time 1 and only the first option seems to work but only from client 1.

On client 1 (Ubuntu 16) the connection is not left open but from client 2 (Ubuntu 18) the connection is left open until it times out. I confirm that either by looking at the target server's watch -n 0.1 "netstat -na | fgrep CLIENT_IP_ADDRESS" or by using -vvv on both clients, which on client 1 is always * Closing connection 0 and on client 2 is always * Connection #0 to host www.example.com left intact.

The difference between client 1 and 2 are obviously the Ubuntu version and curl version. What is it that causes the the connection to be closed in client 1 but not in client 2?

I've also found out that if I change my target server to another machine running an old distro with an old apache httpd, whether I send out Connection: close or not doesn't make any difference and keepalive is always used from either of my 2 clients. So I suppose that the target server configuration also plays some role in that.

Edit: To complicate things even further, if from client 1 and 2 I issue requests to the target server via ab then the connection is killed instantly. That is expected because ab uses HTTP/1.0 and not HTTP/1.1. But then, if I target the old distro with apache then in both cases the connection remains open. Which means that, indeed, the receiving end plays a role as well.

Edit 2: I grabbed all /proc/sys/net/ipv4/ settings of both clients via:

for file in /proc/sys/net/ipv4/*
do
  echo "$file $(cat $file)"
done

and these can be found here:

  • Is there any tcp parameters changes between these 2 clients such as net.ipv4.tcp_syn_retries ? – asktyagi May 21 at 8:58
  • In both clients cat /proc/sys/net/ipv4/tcp_syn_retries returns 6. – cherouvim May 21 at 9:14
  • @asktyagi: I've added all /proc/sys/net/ipv4/ settings in the question. – cherouvim May 21 at 9:37
4

The --no-keepalive option is only useful for TCP keepalive packets, as mentioned in this archived thread on the curl website: https://curl.haxx.se/mail/archive-2013-04/0037.html

Sounds like you'll need to disable HTTP keepalive packets specifically, which would be done on your servers using keepalive_timeout 0; as mentioned in this stackoverflow thread https://stackoverflow.com/questions/24924237/linux-curlnginx-cant-request-with-no-keepalive.

Sorry if the formatting is off, this is my first post :)

Hope that helps!

  • 2
    Your formatting is fine... Welcome to the site :) – kubanczyk May 24 at 16:50
1
+50

Per https://ec.haxx.se/usingcurl-persist.html

curl will always try to keep connections alive and reuse existing connections as far as it can.

Therefore, you cannot use it using the tools as they are.

As @pl-nowlan mentioned in their answer, the only way to do that would be on the server side.

-3

you can use option --no-keepalive. See more at man curl

--no-keepalive Disables the use of keepalive messages on the TCP connection, as by default curl enables them.

Note that this is the negated option name documented. You can thus use --keepalive to enforce keepalive.

  • please read the question. This option has been mentioned. – Sven May 21 at 8:01
  • Thanks for pointing me to the man curl. As I mention in the question I've tried that. Both client 1 and 2 leave the connection open when using --no-keepalive. I've found out that by using -H "Connection: close" at least kills the connection on client 1. – cherouvim May 21 at 8:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.