GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

learn more… | top users | synonyms

50
votes
3answers
47k views

how to download the ssl certificate from a website?

I want to download the ssl certificate from, say https://www.google.com, using wget or any other commands. Any unix command line? wget or openssl?
31
votes
8answers
25k views

Making `wget` not save the page

I'm using the wget program, but I want it not to save the html file I'm downloading. I want it to be discarded after it is received. How do I do that?
14
votes
2answers
1k views

Wget: Is there a way to pass username and password from a file?

Is there a way to pass the username and password from a file instead of the command line via --user and --password? Background: I want to run wget via cron and don't want the username/password show ...
11
votes
5answers
13k views

How do I pipe a downloaded file to standard output in bash?

How is it possible to pipe out wget's downloaded file? If not what alternatives should I use? UPD: Thanks all for your answers, I managed to tune my downloader (curl) to send files to stout. However ...
9
votes
2answers
14k views

Where can I (safely) get libintl3.dll?

I am having problems running the Windows version of wget on a Windows 7 machine. When I run the command I get this error: The program can't start because libintl3.dll is missing from your ...
8
votes
2answers
555 views

Transfer a 30GB tar file from one remote server to another - restrained by disk space

I have a 38GB "Moodle" tar file that I need to transfer from the old server (Shared hosting), which we can call "Server A" to the new server (Dedicated virtual), or "Server B". I'm with Mediatemple if ...
8
votes
1answer
19k views

Accessing a windows shared folder from Linux?

I am trying to copy a (.html) file from a Windows XP Professional shared folder onto a server running Ubuntu Linux 10.04 LTS. As it's a shared folder the usual 'wget' doesn't seem to work. I suspect ...
7
votes
6answers
4k views

wget recursive download, but I don't want to follow all links

I'm trying to mirror a website using wget, but I don't want to download lots of files, so I'm using wget's --reject option to not save all the files. However wget will still download all the files and ...
6
votes
3answers
3k views

Why is only one wget command working in my crontab?

I wish to fetch content from a PHP script on my server two times a day, altering a query variable lang to set what language we want, and save this content in two language specific files. This is my ...
6
votes
4answers
2k views

How to refresh an online website mirror created with `wget --mirror`?

One month ago, I used "wget --mirror" to create a mirror of our public website for temporary use during an upcoming scheduled maintenance window. Our primary website runs HTML, PHP & MySQL, but ...
5
votes
3answers
17k views

Why wget doesn't verify SSL certificates?

I have a problem with my Fedora 8 installation. It looks that wget doesn't know how to verify SSL certificates any more. It's strange because I have another Fedora 8 box which I believe has the same ...
5
votes
5answers
2k views

wget to print errors, but nothing otherwise

How can I have wget print errors, but nothing otherwise? In the default behavior, it shows a progress bar and lots of stuff. In the --no-verbose version still prints one line per downloaded file, ...
5
votes
1answer
4k views

How do I fix certificate errors when running wget on an HTTPS URL in Cygwin Windows?

I have Cygwin installed in my windows system I am trying to execute wget commad for a website but I am gettign the following error message ERROR: The certificate of xxx.com' is not trusted. ERROR: ...
5
votes
1answer
476 views

What to do if wget's progress:mega doen't cut it anymore?

I have script for downloading large files in a queue. Now I would really appreciate it, if I had a decent log. Specifying the -o or -a option is apparently specially made for that, however as of the ...
5
votes
2answers
2k views

wget starts downloading then stops “cannot write to”

I'm using wget to mirror some files across from one server to another. I'm using the following command: wget -x -N -i http://domain.com/filelist.txt -x = Because I want to keep the directory ...
4
votes
4answers
4k views

Wget a script and run it

I want to download a script from: http://dl.dropbox.com/u/11210438/flockonus-stack.sh and execute it. My guess is, to use wget renaming it, chmod it, and execute. What are the commands for doing ...
4
votes
4answers
331 views

How do I reconcile Network Speed from Network Throughput?

I have been arguing with my ISP about the service they are providing; primarily with regard to throughput speeds. I have set up an edge router (see hardware details listed below) and on it I have ...
4
votes
2answers
4k views

Using wget to fetch password protected website (trac wiki page)

I want to write a bash script that will notify me about changes on a trac wiki timeline. Unfortunately I can't "login" (it needs cookie for next logins). wget http://someserver.com/trac/xxx/login \ ...
4
votes
4answers
964 views

wget and pretty urls

In order to automatize stuff, I need to recursively download a web page. I'm using wget as it's probably the most programmer-friendly tool available, using -r flag to trigger link following. wget, ...
3
votes
2answers
2k views

How do I pipe the output of uptime/df to curl?

I want to send the output of uptime and df commands to a web app using cURL. I tried: uptime | curl http://someurl.com -T - But that didn't seem to send the data. What is the proper way to send ...
3
votes
6answers
5k views

How to download with wget without following links with parameters

I'm trying to download two sites for inclusion on a CD: http://boinc.berkeley.edu/trac/wiki http://www.boinc-wiki.info The problem I'm having is that these are both wikis. So when downloading with ...
3
votes
1answer
799 views

“wget --mirror” Alternative

I'm looking for an alternative to using this, it doesn't have to use FTP, but it should provide the same functionality as the "--mirror" option of wget where it only downloads new and changed files. ...
3
votes
4answers
389 views

Download a file from the shell when button click is required

I have a link to a file that I would like to download from the shell. Unfortunately, the URL redirects to a software license agreement page which requires clicking an accept button. This is fine when ...
3
votes
2answers
318 views

pgrep wget: what is the details of process id?

when typing pgrep wget, it show process id 10144 but how to know what is the details of this process id
3
votes
2answers
536 views

Curl always returns same 404 page

No matter what URL I specify for curl I always get the same HTML 404 Error page back. If I use the --verbose option, it looks like curl always connects to the same IP address. $ curl --verbose ...
3
votes
1answer
947 views

How to exclude certain directories while using wget?

I'd like to download a directory from a FTP, which contains some source codes. Initially, I did this: wget -r ftp://path/to/src Unfortunately, the directory itself is a result of a SVN checkout, so ...
3
votes
1answer
935 views

Can wget convert links within a local html document?

I know that wget can fetch a remote page and its dependencies and rewrite the html so that image src attributes reference the newly downloaded images. I am trying to convert local html files that ...
2
votes
3answers
2k views

How to use “wget” to save a file as a different name

wget "http://flakejfl2ke3jrlwkefj38j2ofiwejfoifjalsdkfje3fklwje.exe" is long. How do I save it as my own name?
2
votes
3answers
281 views

wget application replacement

Is there any other application similar to wget? If yes can you post a link?
2
votes
2answers
988 views

what is the default rate unit in wget?

When I use wget to download a large file, it gives me "56M/s". I want to know if it means 56 megabits per second or 56 megabytes per second. Thanks! Like this: 231,997,440 61.4M/s in 4.0s
2
votes
2answers
6k views

Ubuntu 11.10, using wget/curl fails with ssl

On a completely new install of Ubuntu I'm getting the following errors when using wget: wget https://test.sagepay.com --2012-03-27 12:55:12-- https://test.sagepay.com/ Resolving test.sagepay.com... ...
2
votes
2answers
4k views

Alternatives to CURL, WGET

Is anyone aware of any alternatives to curl and wget? The key functional requirement I'm looking for is to be able to execute an HTTPS GET request against a known URL. The reason I'm looking for an ...
2
votes
2answers
7k views

Is there a built-in command-line tool under Windows like wget/curl?

I come from a Linux/Unix background and I have been wondering if Windows has a binary that can download files from the console. I would like to automate a certain process and one of my requirements ...
2
votes
1answer
37 views

Wget Minimum Download Rate

Can anyone help with the parameter that ensures that wget will fetch the file at a minimum specified download rate?
2
votes
1answer
213 views

Downloading from wget - folder issue

I have a website - http://myradio1.localhost and the directory is currently empty. The virtual host works fine, no issues there. This is my command I'm using to download the site via wget: wget ...
2
votes
2answers
499 views

How can i make wget download only pages not css images etc?

I want to download an entire website using wget but I don't want wget to download images, videos etc. I tried wget -bqre robots=off -A.html example.com –user-agent=”Mozilla/5.0 (Windows; U; Windows ...
2
votes
2answers
143 views

how can i use apache log files to recreate usage scenario

Recently i installed a website that had too many requests and it was too slow. Many improvements have been made to the web site code and we've also bought a new server. I want to test the new server ...
2
votes
2answers
954 views

Why is this bash command not echoing into a variable and what can I do to improve?

I've got a bash script I'm working to improve, and have put on a great fix thanks to Dennis Williamson. Unfortunately, one of the lines no longer echoes into a variable I can manipulate, rather dumps ...
2
votes
1answer
216 views

Why does the time it takes to run this script differ so much between user and system and how I might correct the discrepancy?

My application sits behind a load balancer, and every once in a while I like to do a status check on each machine to get an idea of the time it takes to return an index.html document on each machine. ...
2
votes
2answers
853 views

Retrieve FTP directory tree from the command line

What would be the fastest way to recursively retrieve entire directory listing from an ftp server using wget/curl/whatever? I don't need to download any files, just directory and file names. Basically ...
2
votes
1answer
244 views

How to prevent external website from being downloaded in wget?

I'm using wget to download useful website recursively: wget -k -m -r -q -t 1 http://www.web.com/ But external website also downloaded along with the website I want. How to prevent this external ...
2
votes
4answers
2k views

Escape a ! in the password parameter of wget

I'm trying to execute something like this: wget --user=foo --password=bar! url The ! in the password is causing problems. I've tried escaping it with \, as in --password=bar\! I've tried ...
2
votes
2answers
734 views

How to serve a wget --mirror'ed directory of files with questionmarks in them

I'm trying to create a static mirror of a php application (an old php Gallery installation, specifically). The app produces URLs such as: view_album.php?set_albumName=MyAlbum wget downloads these ...
2
votes
2answers
473 views

wget 5xx error code handling

i'm running a shell script that uses wget and pulls something over the web, my question is is there a way to tell wget to abort as soon as it hits 500 error code?
2
votes
6answers
11k views

Using client certificates with wget

I cannot get wget to use the client certificates. The documentation speaks about using the --certificate flag. The use of the certificate flag is clear, I set it to use the PEM version of the client ...
2
votes
1answer
134 views

Find out what's running wget on my server

In my access.log I see requests coming from 127.0.0.1 and retrieving server's "/" and then all of graphics and CSS referenced in it. This happens regularly, every 5 minutes. I want to find out what, ...
2
votes
2answers
2k views

WGet or cURL: Mirror Site from http://site.com And No Internal Access

I have tried wget -m wget -r and a whole bunch of variations. I am getting some of the images on http://site.com, one of the scripts, and none of the CSS, even with the fscking -p parameter. The ...
2
votes
1answer
3k views

HTTP downloads stop after some time, resuming is not possible

When I try to download a file via HTTP, the downloads sometimes stop after around 30 MB. The download rates goes down to 0 B/s and no data keeps coming. When I stop the download and resume again, the ...
2
votes
3answers
76 views

wget and port forwarding

I have a weird problem. I can run this command just fine from various servers: wget --debug '--http-user=USER123' '--http-passwd=PASSWORD' http://GW-BOX:9091/weijhkdsvn/v9_odbc//CRONTAB.2014020 ...
2
votes
2answers
738 views

Reboot if tomcat7 service is not responding

I am running a web app on a Tomcat server. There is a hard-to-detect problem within the server code that causes it to crash once or twice everyday. I will dig in to correct it when I have time. But ...