2

My problem is that I cannot figure out why I'm getting such slow response times using curl and wget. When I say slow, I mean a minimum of five or six seconds for a page which should take only a second or less.

I've tested all kinds of scenarios, but I can't seem to make any difference at all.

My server is: CentOS Linux release 7.3.1611 (Core)

I'm trying to fix this because it is slowing down a newly installed WordPress application. I ended up identifying (using Xdebug) that, rather than the problem being WP, it appears to be PHP calls which use curl. Subsequently I've been testing from the command line, without any other application.

So, for example:

$ curl -o /dev/null "http://techrepublic.com"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
  100   178  100   178    0     0     30      0  0:00:05  0:00:05 --:--:--    41

It doesn't make any difference what the URL is, but if I use an IP address, instead of a domain name, then there's no problem at all.

The same problem is also true using wget.

So, some of the things I've tried are:

  • a) Changing the DNS servers in /etc/resolv.conf
  • b) Same effect with curl and also with wget
  • c) Same effect with both http and also with https
  • d) Same effect with both heavy and lightweight pages
  • e) Tested with curl on Windows 10, on same network, with same router, etc, and it takes about a second or less
  • f) Tested as a regular user, and as root

My primary DNS server is now listed in /etc/resolv.conf as 8.8.8.8, which should be fine.

To get more information, I also used this:

$ curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect:%{time_connect}\nappconnect:%{time_appconnect}\npretransfer:%{time_pretransfer}\nredirect:%{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal:%{time_total}\n' 'techrepublic.com'
Output:
lookup: 5.514
connect:5.528
appconnect:0.000
pretransfer:5.528
redirect:5.819
starttransfer: 6.680
total:12.552

(not sure why, but using curl this way seems to take longer than using it without all the options, but either way it's still taking far too long)

So, I'm figuring that somehow, somewhere, this is related to DNS lookups, but I'm really not sure what else to try. It's very frustrating.

Any assistance would be greatly appreciated. Thanks!

3
  • What you see when try to use nslookup? Jun 29, 2017 at 22:01
  • Keep in mind that changing the IP address might also change how the content is rendered. By not supplying a hostname, you would land in the default virtual host instead of the name based virtual host associated with that FQDN. (assuming they are different) You might want to start by making sure that the output of both commands is identical...in your example both are being sent to /dev/null. IPv6 transport problems are also common in this situation, but techrepublic.com does not have a AAAA record.
    – Andrew B
    Jun 29, 2017 at 22:50
  • Thanks for the comments. In fact I didn't notice them until after I found a solution (as posted below), but I appreciate your thoughts.
    – SuperDog
    Jun 30, 2017 at 0:23

1 Answer 1

4

Okay, problem solved. After delving into the web again for similar problems I came upon this site, which described the same issue:

http://aarvik.dk/disable-ipv6/

Adding the following line to the bottom of /etc/resolv.conf resolved the issue for me, but I recommend you read the article in full:

options single-request-reopen
1
  • Thanks for sharing your solution. This fix worked perfectly for me using Centos 7 under Parallels. Jan 3, 2019 at 19:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .