Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have some links such as:

  • http:/foo.com/1/1/1/1.jpg
  • http:/foo.com/1/2/2/2.jpg
  • http:/foo.com/1/3/3/3.jpg
  • ...

How I can I download all files and directories with wget?

share|improve this question

closed as off topic by EEAA, womble, Ward, John Gardeniers, Khaled May 13 '12 at 7:23

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

2 Answers 2

If you have a list of urls in a file named links.txt:

for url in $(cat links.txt); do wget $url; done

With aria2:

aria2c -i links.txt

If what you have is a html file with lots of href and other tags..., you could us regexp to parse them and finally get a clean list.

share|improve this answer

HTTP doesn't really expose a filesystem, so wget typically can't just grab the whole directory etc. It can only work with resources it knows about. It will try to grab each URL that is an href or src of the appropriate elements in the page/site you point it at, but if the files aren't linked to from the page, wget doesn't even know about them, so it won't try to get them.

Translation: If you want to get all that stuff, have it linked from somewhere in the page/site. Or, use FTP, which is far better suited for the job.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.