0

My problem is that curl and wget usually fail to retrieve more than just the first part of any given file, and often nothing at all. Both have worked previously on the same server, and nothing else on my network has changed - and I'm not really sure what's changed on my server to now make them not work, although very intermittently it will work just fine.

I have automatic updates turned on, so maybe something to do with that (but maybe not), but either way I have no idea how to resolve.

So, my server is running CentOS Linux release 7.3.1611 (update: now updated to 7.6.1810)

My version of curl is very recent:

curl 7.64.0 (x86_64-redhat-linux-gnu) libcurl/7.64.0 NSS/3.36 zlib/1.2.7 libpsl/0.7.0 (+libicu/50.1.2) libssh2/1.8.0 nghttp2/1.31.1

The issue was first apparent to me when I was trying to do WordPress updates (and it's been an issue for a while), when updates will fail with, for example:

Download failed. cURL error 28: Operation timed out after 300000 milliseconds with 1355444 out of 5007255 bytes received

Occasionally the updates will work, but I think that's generally with very small files.

So, with regards to trying to resolve the issue with WordPress, I tried:

  • increased the timeout in Wordpress files: class-wp-http-curl.php and class-http.php
  • increased max_execution_time in /etc/php.ini to 120

... but no change. Also tried from the command line using both curl and wget, but same issue.

e.g. $ curl -O -v https://downloads.wordpress.org/plugin/nimble-builder.1.5.0.zip
....
curl: (56) I/O operation timed out

Also:

a) resolv.conf - previously had added: options single-request-reopen, which resolved a conflict with IPv4/IPv6 many months ago
- also used several different DNS server setups
b) tried fetching stuff from both http and https sites
c) tried fetching using IP addresses instead of domain names
d) firewall - temporarily prevented iptables from blocking incoming connections:

$ iptables -I INPUT 1 -j ACCEPT
and then restored after testing:
$ iptables -D INPUT 1

e) rebooted
f) all OS updates are installed

Not quite sure what else to try. It's very frustrating. Any assistance appreciated. Thanks!

Update-1: I can use yum with no problems, and have subsequently updated the system, and it's now running CentOS Linux release 7.6.1810, but still I have the same problems with curl and wget.

Update-2: Sample strace output is given below:

Near beginning, a lot like this:
open("/lib64/libk5crypto.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220H\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=210824, ...}) = 0
mmap(NULL, 2302664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff0cbf8c000
mprotect(0x7ff0cbfbd000, 2093056, PROT_NONE) = 0
mmap(0x7ff0cc1bc000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x30000) = 0x7ff0cc1bc000
close(3)                                = 0

Part way through, stuff like this:
access("/home/xyz/.pki/nssdb/.57765795_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765796_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765797_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765798_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765799_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765800_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
... 
and quite a few of these:
recvfrom(3, 0x184e0d5, 12115, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}], 1, 1000)  = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=3, revents=POLLIN|POLLRDNORM}])
recvfrom(3, "\322U\200l9\332+\31w\235*\322dt\233\273\37]\244~\264{\24g\2\301\320\276\271\207\337["..., 12115, 0, NULL, NULL) = 1448
recvfrom(3, 0x184e67d, 10667, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)

Close to end, a lot like this:
write(2, "k", 1)                        = 1
poll([{fd=3, events=POLLIN}], 1, 1000)  = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
write(2, "\r", 1)

Memory:

              total        used        free      shared  buff/cache   available
Mem:        8175404     1734268      310528       20832     6130608     5936760
Swap:       8257532       85472     8172060

Update-3: I have also now disabled IPv6, in case of any conflicts, but that too makes no difference.

Update-4: I've since found that yum (which works) uses Python, whereas curl and wget (which do not work) do not use Python. Using a little Python script to test the same URLs I've tried to retrieve using curl/wget works just fine, so I'm guessing there must be some library/code/something which curl/wget use, which Python does not, which isn't playing very nicely. However, I've no idea how to research/test/etc, such that I can get curl/wget working. Still, at least I do know that it's not a million other things which it could have been, but would be very grateful if someone can assist me in resolving. Thanks!

  • "7.3.1611"? You don't have all OS updates; you're several years behind. Though I suppose with your networking problem you're not going to be able to update until you fix it. What exactly is this server? Where is it hosted? What is unusual about the server and network? – Michael Hampton Mar 7 at 22:57
  • This could also be an issue with the server's I/O systems, that is hard disk. – Tero Kilkanen Mar 7 at 22:59
  • @Michael: Server is hosted in my office on a dedicated machine; used as a database server and web server and for dev; nothing unusual about it, or the network, as far as I know. Also, I can use yum with no problems, it seems, and I've updated the system; now running CentOS Linux release 7.6.1810. (but still the same problems with curl). – SuperDog Mar 8 at 1:21
  • 1
    @eckes. I'm not too familiar with using strace or understanding the output from it, but have updated the question with sample strace output. In fact a couple of times I've tried downloading today (and previously) it's worked just fine, and quickly, but 95% of the time it will either download nothing, or just the first bit. – SuperDog Mar 8 at 13:02
  • 1
    @eckes. Should be no memory issues; question updated with memory details. – SuperDog Mar 8 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.