Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am using cURL to try to download all files in a certain directory.

Here's what my list of files looks like:

List of Files

I have tried to do in bash script: iiumlabs.[].csv.pgp and iiumlabs* and I guess cURL is not big on wildcards.

curl -u login:pass ftp.myftpsite.com/iiumlabs* -O

Question: How do I download this directory of files using cURL?

share|improve this question
    
Why not use wget with -r and maybe some other options instead? curl can be made to do it with some scripting around it, but there are other tools that are better suited. –  cjc Aug 2 '12 at 20:01
    
Why not use good old ftp with a script file to mget all the files? –  Chris S Aug 2 '12 at 20:28
    
@ChrisS thank you, can i do sFTP with mget? –  Yuck Aug 2 '12 at 21:41
    
@cjc thank you! how would i do sftp with wget? –  Yuck Aug 2 '12 at 21:41
    
Is the server SFTP only, or can you login with SSH? If the server isn't limited to SFTP only, you would probably be better off using scp/rsync. –  Zoredache Aug 2 '12 at 22:14
show 3 more comments

1 Answer

up vote 1 down vote accepted

Curl does not support recursive download. Use wget --mirror --no-parent [URL]

EDIT: For SSH, from the man page of curl:

Get a file from an SSH server using SFTP:

    curl -u username sftp://shell.example.com/etc/issue

Get a file from an SSH server using SCP using a private key to authenticate:

    curl -u username: --key ~/.ssh/id_dsa --pubkey ~/.ssh/id_dsa.pub \
            scp://shell.example.com/~/personal.txt
share|improve this answer
    
great! what would be the syntax for sftp (transfer over ssh) ? –  Yuck Aug 2 '12 at 21:40
    
thank yuou very much!!!!!! what is SCP? scp://..... if the key is stored on my machine can i just point to c:/..... –  Yuck Aug 3 '12 at 23:25
    
scp is an old way to use ssh to transfer files. The latest version of ssh support SFTP. I do not know the syntax for local files in curl. You can enable some debugging options in curl. Besides curl you can use pscp and psftp to transfer files. See: chiark.greenend.org.uk/~sgtatham/putty/download.html. –  Mircea Vutcovici Aug 4 '12 at 0:30
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.