Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am using cURL to try to download all files in a certain directory.

Here's what my list of files looks like:

List of Files

I have tried to do in bash script: iiumlabs.[].csv.pgp and iiumlabs* and I guess cURL is not big on wildcards.

curl -u login:pass* -O

Question: How do I download this directory of files using cURL?

share|improve this question
Why not use wget with -r and maybe some other options instead? curl can be made to do it with some scripting around it, but there are other tools that are better suited. – cjc Aug 2 '12 at 20:01
Why not use good old ftp with a script file to mget all the files? – Chris S Aug 2 '12 at 20:28
@ChrisS thank you, can i do sFTP with mget? – l--''''''---------'''''''''''' Aug 2 '12 at 21:41
@cjc thank you! how would i do sftp with wget? – l--''''''---------'''''''''''' Aug 2 '12 at 21:41
Is the server SFTP only, or can you login with SSH? If the server isn't limited to SFTP only, you would probably be better off using scp/rsync. – Zoredache Aug 2 '12 at 22:14
up vote 1 down vote accepted

Curl does not support recursive download. Use wget --mirror --no-parent [URL]

EDIT: For SSH, from the man page of curl:

Get a file from an SSH server using SFTP:

    curl -u username s

Get a file from an SSH server using SCP using a private key to authenticate:

    curl -u username: --key ~/.ssh/id_dsa --pubkey ~/.ssh/ \
share|improve this answer
great! what would be the syntax for sftp (transfer over ssh) ? – l--''''''---------'''''''''''' Aug 2 '12 at 21:40
thank yuou very much!!!!!! what is SCP? scp://..... if the key is stored on my machine can i just point to c:/..... – l--''''''---------'''''''''''' Aug 3 '12 at 23:25
scp is an old way to use ssh to transfer files. The latest version of ssh support SFTP. I do not know the syntax for local files in curl. You can enable some debugging options in curl. Besides curl you can use pscp and psftp to transfer files. See: – Mircea Vutcovici Aug 4 '12 at 0:30

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.