Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How can I download multiple files stored in a text file with curl and xargs? This is my last trial:

cat listfile.txt | xargs curl -O

first file works well, but other files are just output to stdout.

share|improve this question

Using GNU Parallel you can do:

cat listfile.txt | parallel curl -O

Not only does GNU Parallel deal nicely with special chars like ' " and space, you will also get the added benefit of downloading in parallel.

Watch the intro video to GNU Parallel:

share|improve this answer
Nice suggestion. But I have no time ti test this, so +1, and thanks! – Eonil Aug 27 '10 at 1:31
This is awesome. – Mr.Gando Jun 27 '13 at 16:21
up vote 1 down vote accepted

I found solution:

cat ./../c | xargs -n1 curl -O

xargs splits stdin by spaces and newlines, and passes to curl at once. So curl called only once with long arguments.

n1 option limits this passing argument count as 1, so curl will be called multiple times.

share|improve this answer

xargs doesn't know what curl is so it can't determine how many arguments it should pass in one round. So the solution is to pass a -n1 option to it as you mentioned.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.