Tag Info

Hot answers tagged

13

curl already uses keepalive by default. As an example: curl -v http://www.google.com http://www.google.com Produces the following: * About to connect() to www.google.com port 80 (#0) * Trying 74.125.39.99... connected * Connected to www.google.com (74.125.39.99) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) ...


11

The error is fairly clear. Essentially what is happening is that the cert on the remote end is either self signed, or signed by an authority that wget isn't recognizing. You can pass the following command while invoking wget to get it to ignore cert errors: --no-check-certificate


11

Curl is using the system-default CA bundle is stored in /etc/pki/tls/certs/ca-bundle.crt . Before you change it, make a copy of that file so that you can restore the system default if you need to. You can simply append new CA certificates to that file, or you can replace the entire bundle. Are you also wondering where to get the certificates? I (and ...


9

It seems the difference has to do with the Content-Length header and how it is treated by both commands. But before going into that, curl -X HEAD does not give any output because, by default, curl does not print headers if switch -i is not provided (not needed on -I though). In any case, curl -I is the proper way to fetch the headers. It just ask for the ...


8

From what is mentioned in http://pecl.php.net/bugs/bug.php?id=17012&edit=1, pecl does not recognize php modules that were not statically compiled into php. So if you are using any shared libraries, pecl will ignore them. Pecl also ignores your php.ini. The good news is there are still two easy ways to install this if you have curl as a shared ...


8

I was having this exact problem (today as well, found your post while searching for a result) The following worked for me: cd /Library/Caches/Homebrew curl -O http://fossies.org/unix/privat/pkg-config-0.27.tar.gz brew install imagemagick Basically Homebrew will not re-download a file that it has cached, so its just about downloading the file from ...


7

You can use the -d option in curl with a @- argument to accept input from a pipe. You will need to construct the key-value pairs yourself. Try this: echo "time=`uptime`" | curl -d @- http://URL The backticks (`) denote that the enclosed command (in this case uptime) should be executed, and the backtick-quoted text replaced with the output of the executed ...


6

I think this is a bug in curl. If I specify a method with -X, curl should handle the response according to the RFC. Unfortunately, the maintainer of curl does not agree. Someone filed a bug and even submitted a patch: http://sourceforge.net/tracker/?func=detail&atid=100976&aid=1810273&group_id=976 but the curl maintainer rejected it. Apparently ...


6

wget(1) works sequentally by default, and has this option built in: -i file --input-file=file Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.) If this function is used, no URLs need be present on the command line. If there are ...


5

Had a similar problem with Ubuntu 12.04 running curl from the command line to get aws command-line tools which were in github. I found if I ran the command-line as such: curl https://raw.github.com/timkay/aws/master/aws -o aws --cacert /etc/ssl/certs/ca-certificates.crt it would work. But trying to set the CURL_CA_BUNDLE environment variable so I could ...


5

Using GNU Parallel http://www.gnu.org/software/parallel/ you can do: cat listfile.txt | parallel curl -O Not only does GNU Parallel deal nicely with special chars like ' " and space, you will also get the added benefit of downloading in parallel. Watch the intro video to GNU Parallel: http://www.youtube.com/watch?v=OpaiGYxkSuQ


5

You can use tcpdump to sniff some packets on the server, something like this: # tcpdump -vv -s0 tcp port 80 -w /tmp/apache_outgoing.pcap and run your PHP script to see what happens. Is there any way to restrict it to a) Only POST data, You can sniff all and filter with http.request.method == POST in Wireshark. b) Only coming from 1.1.1.1 # ...


5

[Pushing up from comments] Two options: Batch mode from text-based browsers such as links/lynx/w3m Use openssl s_client as noted here: Connecting to HTTPS with netcat (nc)


5

curl -I will generate a HEAD request. Adding the -L option will make it follow redirects. Timeouts can be controlled with several options depending on which bit of the request you want to time out. From the man page: --connect-timeout <seconds> Maximum time in seconds that you allow the connection to the server to take. This only limits ...


4

So it turns out that installing the ca-certificates package didn't install the one that I needed. I found this post about certificates being presented out of order. This seems to be the case with my request to sagepay. The solution ended up being to install another CA certificate from Verisign. I'm not sure why this fixes the issue with it being out of ...


4

As everybody has noted, doing this is nearly impossible and trying gains you nothing. Don't waste your time. You will only hurt yourself by trying to limit access methods. If your goal is to distribute content, welcome to the web. If it's not, fighting it is a losing battle. If you have subscription content, consider proper methods including user ...


4

* Connected to www.edgeoftheweb.co.uk (::ffff:74.117.222.24) port 80 * Connected to api.twitter.com (::ffff:74.117.222.24) port 80 Seems to me curl is using ipv6 to connect while wget is using ipv4 Try the following curl --verbose -4 http://api.twitter.com


4

instead of using wget, curl works like this: curl --silent http://domain.com/cron.php which will work in the same way as wget. if its a php file you are launching, is there any reason you cant run it via the command line php interpreter like so: php -q /path/to/cron.php same on a webserver request and often will work much faster and without certain ...


4

Depends on the nature of the networking as far as your server is concerned, and what it would be connecting to. In most cases, you'll lose two addresses out of a full 256 octet - one for the network address (192.168.21.0) and one for the broadcast address (192.168.21.255). Remember that you'll also need a router in the same local subnet (192.168.21.1), so ...


4

I think FOG ( http://www.fogproject.org ) meets your requirements, boots over the network, restores images from a central repository. Currently trying to transition to it from PXE booted Norton Ghost here. You might also take a look at Clonezilla - http://clonezilla.org


4

Squid doesn't support Expect: 100-continue, being a HTTP/1.0 device. Notice the response status code 417 Expectation failed. Curl should retry the request automatically, but it doesn't; instead, you need to suppress the Expect header; e.g., curl -H "Expect:" --trace multipart-fail.log -F "source={}" http://127.0.0.1:3003/jslint


4

This is possible using curl within a shell script, something like this but you'll need to research appropriate options for curl etc for yourself while read URL curl some options $URL if required check exit status take appropriate action done <fileontainingurls


4

In your https virtual host, you specified an IP address to listen for connections on. Thus, requests which connect to any other IP address on the host will never match that virtual host. If you really wanted to do this, you could add a second IP address to the <VirtualHost> declaration, or just make it a wildcard *:443.


4

Add -v to curl to see the actual header responses. And by default it doesn't follow redirects, add -L to also follow redirects, but (from the man page): When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won't be able to intercept the user+password. See also ...


4

You can use bash's process substitution >(cmd). For example: curl -s -w "%{http_code}\n" -o >(ruby -ne 'printf("-%6s%s", $., $_)') http://www.example.com/ Another possibility that may suit your needs is curl -v, which will send the headers to stderr and the body to stdout. E.g., curl -v http://www.example.com/ | ruby -ne 'printf("-%6s%s", $., $_)' ...


4

Instead of explicitly trusting the intermediate certificates, you should have your web server send the full certificate chain so that a client's trust of the root (not the intermediate) will allow them to validate. Importing the intermediate manually may fix it on your system, but other people using that OS or others that don't have the intermediate (often ...


4

Probably depends which version of Redhat. You can find which package actually updates the file by doing: rpm -qf /etc/pki/tls/certs/ca-bundle.crt My result was showing that openssl-0.9.8e-12.el5 needs to be updated. If there is no updated certificates in your distribution, you have to manually update, as per Nada's answer.


4

As @Zoredache points out, ssh relays the status of the remote command as its own exit status, so error detection works transparently over SSH. However, two important points require special consideration in your example. First, curl tends to be very lenient, treating many abnormal conditions as success. For example, curl ...


3

We were able to resolve this by disabling ipv6. We disabled ipv6 by adding the following to /etc/modprobe.conf and rebooting. alias net-pf-10 off alias ipv6 off options ipv6 disable=1 After reboot, we no longer see DNS lookups for hosts listed in /etc/hosts. It's not clear to me exactly why this resolves the issue.


3

You seem to have a typo. $ openssl s_client -connect evernote.com:433 Port 433 is not open on the remote host, which is why you got error 110 (Connection refused). Try using port 443 (the standard HTTPS port) instead. As for curl, I can replicate the problem when using evernote.com, but www.evernote.com works properly (though it sends a 302 redirect ...



Only top voted, non community-wiki answers of a minimum length are eligible