Hot answers tagged

33

For RHEL 6 or later, you should be using update-ca-trust, as lzap describes in his answer below. --- For older versions of Fedora, CentOS, Redhat: Curl is using the system-default CA bundle is stored in /etc/pki/tls/certs/ca-bundle.crt . Before you change it, make a copy of that file so that you can restore the system default if you need to. You can ...


24

It seems the difference has to do with the Content-Length header and how it is treated by both commands. But before going into that, curl -X HEAD does not give any output because, by default, curl does not print headers if switch -i is not provided (not needed on -I though). In any case, curl -I is the proper way to fetch the headers. It just ask for the ...


24

Think I found a solution going through the cURL manual: curl https:// DOMAIN.TLD --resolve 'DOMAIN.TLD:443:IP_ADDRESS'


24

You should be able to use OpenSSL for your purpose: echo | openssl s_client -showcerts -servername gnupg.org -connect gnupg.org:443 2>/dev/null | openssl x509 -inform pem -noout -text That command connects to the desired website and pipes the certificate in PEM format on to another openssl command that reads and parses the details. (Note that "...


21

Yeah, you can do that, as curl --help or man curl would have told you: -k, --insecure (SSL) This option explicitly allows curl to perform "insecure" SSL connections and transfers. All SSL connections are attempted to be made secure by using the CA certificate bundle installed by default. This makes all connections considered "insecure" fail ...


20

curl already uses keepalive by default. As an example: curl -v http://www.google.com http://www.google.com Produces the following: * About to connect() to www.google.com port 80 (#0) * Trying 74.125.39.99... connected * Connected to www.google.com (74.125.39.99) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7....


18

You can use the -d option in curl with a @- argument to accept input from a pipe. You will need to construct the key-value pairs yourself. Try this: echo "time=`uptime`" | curl -d @- http://URL The backticks (`) denote that the enclosed command (in this case uptime) should be executed, and the backtick-quoted text replaced with the output of the executed ...


16

The recommended way of doing that on RHEL 6+ systems is to use update-ca-trust tool, which is now installed by default. # cat /etc/pki/ca-trust/source/README This directory /etc/pki/ca-trust/source/ contains CA certificates and trust settings in the PEM file format. The trust settings found here will be interpreted with a high priority - higher than the ...


15

Powershell. $wc = New-Object System.Net.WebClient $wc.DownloadFile($source, $dest) There's also Invoke-WebRequest in PS 3.0.


12

Use '&' after a command to background a process, and 'wait' to wait for them to finish. Use '()' around the commands if you need to create a sub-shell. #!/bin/bash curl -s -o foo http://example.com/file1 && echo "done1" & curl -s -o bar http://example.com/file2 && echo "done2" & curl -s -o baz http://example.com/file3 &&...


12

wget(1) works sequentally by default, and has this option built in: -i file --input-file=file Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.) If this function is used, no URLs need be present on the command line. If there are ...


11

The error is fairly clear. Essentially what is happening is that the cert on the remote end is either self signed, or signed by an authority that wget isn't recognizing. You can pass the following command while invoking wget to get it to ignore cert errors: --no-check-certificate


11

You are correct -- lighttpd doesn't support POST requests with an empty message body without a 'Content-Length' header set to zero, and CURL sends such a request. There's argument back and forth about who's right, but in my opinion, lighttpd is broken. A POST with no Content-Length and no Transfer-Encoding is perfectly legal and has no message body. Adding -...


11

Your curl command is running as root (via sudo) but your php command is not. Try moving your sudo between the pipe ("|") and 'php' (assuming you want your installer to run as root in the first place...).


10

I think this is a bug in curl. If I specify a method with -X, curl should handle the response according to the RFC. Unfortunately, the maintainer of curl does not agree. Someone filed a bug and even submitted a patch: http://sourceforge.net/tracker/?func=detail&atid=100976&aid=1810273&group_id=976 but the curl maintainer rejected it. Apparently ...


10

I've been having the same trouble and after poking around a bit found that you can download a package of CA-certs ready for curl on ubuntu directly from the curl dev site. cd /etc/ssl/certs sudo wget http://curl.haxx.se/ca/cacert.pem Now curl uses the most up-to-date bundle and you're good to go.


10

I think your problem is DNSSEC-related. There exists a DS record for the zone: [me@risby ~]$ dig ds codestronaut.com [...] ;; ANSWER SECTION: codestronaut.com. 86363 IN DS 19465 8 1 42AFC90FE0A61D3993051C55B6C4C35518713921 but the A record for chat is unsigned: [me@risby ~]$ dig +dnssec +trace chat.codestronaut.com [...] codestronaut....


9

Had a similar problem with Ubuntu 12.04 running curl from the command line to get aws command-line tools which were in github. I found if I ran the command-line as such: curl https://raw.github.com/timkay/aws/master/aws -o aws --cacert /etc/ssl/certs/ca-certificates.crt it would work. But trying to set the CURL_CA_BUNDLE environment variable so I could ...


9

Using GNU Parallel http://www.gnu.org/software/parallel/ you can do: cat listfile.txt | parallel curl -O Not only does GNU Parallel deal nicely with special chars like ' " and space, you will also get the added benefit of downloading in parallel. Watch the intro video to GNU Parallel: http://www.youtube.com/watch?v=OpaiGYxkSuQ


9

From what is mentioned in http://pecl.php.net/bugs/bug.php?id=17012&edit=1, pecl does not recognize php modules that were not statically compiled into php. So if you are using any shared libraries, pecl will ignore them. Pecl also ignores your php.ini. The good news is there are still two easy ways to install this if you have curl as a shared ...


9

instead of using wget, curl works like this: curl --silent http://domain.com/cron.php which will work in the same way as wget. if its a php file you are launching, is there any reason you cant run it via the command line php interpreter like so: php -q /path/to/cron.php same on a webserver request and often will work much faster and without certain ...


9

No, cURL is not a standard part of operating systems. It isn't ever standard on all Linux based distributions.


8

curl http://google.com > /path/to/dir/index.html


8

I was having this exact problem (today as well, found your post while searching for a result) The following worked for me: cd /Library/Caches/Homebrew curl -O http://fossies.org/unix/privat/pkg-config-0.27.tar.gz brew install imagemagick Basically Homebrew will not re-download a file that it has cached, so its just about downloading the file from ...


8

curl -k or curl --insecure does NOT fix this particular error condition: "curl: (51) SSL peer certifcate


8

I saw a similar kind of problem, yet not accessing twitter. Problem I saw was on RHEL 6.1 -> RHEL 6.4, using curl to access a REST API. curl showed error: curl: (35) SSL connect error Issue ended up being that our server running the REST API was RHEL 6.6, using a newer version of NSS. Once we upgraded the client NSS version curl was successful. We ...


8

I suspect that any performance increases you'll see are from improving whatever wrapper you're using to make your connections, rather than the overhead of launching curl for each URL. Whether it's curl or netcat or wget, you'll probably want to launch each one separately in order to process their results separately. But I'll answer this question in two ...


7

You can also use the --insecure option to bypass SSL verification


7

install the package apt-get install php5-curl at this point php should already be configured to load that library [ /etc/php5/apache2/conf.d/curl.ini ]; restart apache /etc/init.d/apache2 restart


7

Have a look at A.J. Clark's blog: xargs -P 20 -n 1 wget -nv <urs.txt



Only top voted, non community-wiki answers of a minimum length are eligible