GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

learn more… | top users | synonyms

102
votes
5answers
132k views

how to download the ssl certificate from a website?

I want to download the ssl certificate from, say https://www.google.com, using wget or any other commands. Any unix command line? wget or openssl?
44
votes
8answers
42k views

Making `wget` not save the page

I'm using the wget program, but I want it not to save the html file I'm downloading. I want it to be discarded after it is received. How do I do that?
25
votes
5answers
16k views

Piping wget output to /dev/null in cron

I am running the following command every 5 minutes in my crontab to keep Phusion Passenger alive. */5 * * * * wget mysite.com > /dev/null 2>&1 When I run this it performs a wget on the ...
17
votes
6answers
25k views

How do I pipe a downloaded file to standard output in bash?

How is it possible to pipe out wget's downloaded file? If not what alternatives should I use?
15
votes
3answers
6k views

Wget: Is there a way to pass username and password from a file?

Is there a way to pass the username and password from a file instead of the command line via --user and --password? Background: I want to run wget via cron and don't want the username/password show ...
12
votes
3answers
7k views

Why is only one wget command working in my crontab?

I wish to fetch content from a PHP script on my server two times a day, altering a query variable lang to set what language we want, and save this content in two language specific files. This is my ...
12
votes
4answers
47k views

Is there a built-in command-line tool under Windows like wget/curl?

I come from a Linux/Unix background and I have been wondering if Windows has a binary that can download files from the console. I would like to automate a certain process and one of my requirements ...
12
votes
5answers
6k views

wget to print errors, but nothing otherwise

How can I have wget print errors, but nothing otherwise? In the default behavior, it shows a progress bar and lots of stuff. In the --no-verbose version still prints one line per downloaded file, ...
12
votes
2answers
17k views

Where can I (safely) get libintl3.dll?

I am having problems running the Windows version of wget on a Windows 7 machine. When I run the command I get this error: The program can't start because libintl3.dll is missing from your computer....
11
votes
5answers
16k views

Wget a script and run it

I want to download a script from: http://dl.dropbox.com/u/11210438/flockonus-stack.sh and execute it. My guess is, to use wget renaming it, chmod it, and execute. What are the commands for doing ...
11
votes
1answer
45k views

Accessing a windows shared folder from Linux?

I am trying to copy a (.html) file from a Windows XP Professional shared folder onto a server running Ubuntu Linux 10.04 LTS. As it's a shared folder the usual 'wget' doesn't seem to work. I suspect ...
11
votes
3answers
38k views

Why wget doesn't verify SSL certificates?

I have a problem with my Fedora 8 installation. It looks that wget doesn't know how to verify SSL certificates any more. It's strange because I have another Fedora 8 box which I believe has the same ...
9
votes
2answers
1k views

Transfer a 30GB tar file from one remote server to another - restrained by disk space

I have a 38GB "Moodle" tar file that I need to transfer from the old server (Shared hosting), which we can call "Server A" to the new server (Dedicated virtual), or "Server B". I'm with Mediatemple if ...
9
votes
6answers
67k views

How to allow wget to overwrite files

Using wget command, how do I allow/instruct to overwrite my local file everytime, irrespective of how many times I invoke. Let's say, I want to download a file from the location: http://server/folder/...
8
votes
6answers
8k views

wget recursive download, but I don't want to follow all links

I'm trying to mirror a website using wget, but I don't want to download lots of files, so I'm using wget's --reject option to not save all the files. However wget will still download all the files and ...
7
votes
3answers
15k views

Execute curl requests in parallel in bash

What is the best way to execute 5 curl requests in parallel from a bash script? I can't run them in serial for performance reasons.
7
votes
1answer
10k views

How do I pipe the output of uptime/df to curl?

I want to send the output of uptime and df commands to a web app using cURL. I tried: uptime | curl http://someurl.com -T - But that didn't seem to send the data. What is the proper way to send ...
7
votes
3answers
7k views

wget starts downloading then stops “cannot write to”

I'm using wget to mirror some files across from one server to another. I'm using the following command: wget -x -N -i http://domain.com/filelist.txt -x = Because I want to keep the directory ...
6
votes
6answers
8k views

How to download with wget without following links with parameters

I'm trying to download two sites for inclusion on a CD: http://boinc.berkeley.edu/trac/wiki http://www.boinc-wiki.info The problem I'm having is that these are both wikis. So when downloading with ...
6
votes
1answer
13k views

How do I fix certificate errors when running wget on an HTTPS URL in Cygwin Windows?

I have Cygwin installed in my windows system I am trying to execute wget commad for a website but I am gettign the following error message ERROR: The certificate of xxx.com' is not trusted. ERROR: ...
6
votes
4answers
4k views

How to refresh an online website mirror created with `wget --mirror`?

One month ago, I used "wget --mirror" to create a mirror of our public website for temporary use during an upcoming scheduled maintenance window. Our primary website runs HTML, PHP & MySQL, but ...
5
votes
7answers
56k views

WGET can't resolve host

I've got 2 machines on the same sub-net running Debian 5.0. They use the same DNS (/etc/resolv.conf), they route to the same gateway (#route) and they have the same iptables settings (#iptables -L). ...
5
votes
3answers
31k views

How to install wget on this?

I did download RubyStack 2.0.3 for VMWare (Ubuntu 9.10) but I cannot download anything on it! It appears that all basic utilities are missing/screwed: bitnami@linux:/var/tmp$ wget -bash: wget: ...
5
votes
2answers
877 views

What to do if wget's progress:mega doen't cut it anymore?

I have script for downloading large files in a queue. Now I would really appreciate it, if I had a decent log. Specifying the -o or -a option is apparently specially made for that, however as of the ...
5
votes
2answers
5k views

Using wget to fetch password protected website (trac wiki page)

I want to write a bash script that will notify me about changes on a trac wiki timeline. Unfortunately I can't "login" (it needs cookie for next logins). wget http://someserver.com/trac/xxx/login \ --...
5
votes
4answers
1k views

wget and pretty urls

In order to automatize stuff, I need to recursively download a web page. I'm using wget as it's probably the most programmer-friendly tool available, using -r flag to trigger link following. wget, ...
5
votes
1answer
4k views

curl and wget attempting an ipv6 connection

On my dedicated server (hosted by OVH), that is running a fresh install of Ubuntu 14.04, curl and wget take approximately 10 seconds to complete a simple request. $ curl -v google.com * Rebuilt URL ...
4
votes
3answers
6k views

How to use “wget” to save a file as a different name

wget "http://flakejfl2ke3jrlwkefj38j2ofiwejfoifjalsdkfje3fklwje.exe" is long. How do I save it as my own name?
4
votes
1answer
1k views

wget receives the file and hangs [closed]

I am trying to beat weird problem - wget gets the file, saves it to disk and hangs. Here is the details: wget --server-response --ca-directory=/etc/ssl/certs --no-dns-cache -T 1 --read-timeout=1 --...
4
votes
2answers
3k views

what is the default rate unit in wget?

When I use wget to download a large file, it gives me "56M/s". I want to know if it means 56 megabits per second or 56 megabytes per second. Thanks! Like this: 231,997,440 61.4M/s in 4.0s
4
votes
5answers
5k views

Multithreaded downloading with shell script

Let's say I have a file with lots of URLs and I want to download them in parallel using arbitrary number of processes. How can I do it with bash?
4
votes
4answers
19k views

curl succeeds on an https request, wget does not

When I run curl from the command line curl "https://example.com" It succeeds immediately, returning the result of the request. When I run the same wget command wget https://example.com It ...
4
votes
6answers
26k views

Using client certificates with wget

I cannot get wget to use the client certificates. The documentation speaks about using the --certificate flag. The use of the certificate flag is clear, I set it to use the PEM version of the client ...
4
votes
4answers
771 views

How do I reconcile Network Speed from Network Throughput?

I have been arguing with my ISP about the service they are providing; primarily with regard to throughput speeds. I have set up an edge router (see hardware details listed below) and on it I have ...
4
votes
2answers
14k views

Alternatives to CURL, WGET

Is anyone aware of any alternatives to curl and wget? The key functional requirement I'm looking for is to be able to execute an HTTPS GET request against a known URL. The reason I'm looking for an ...
4
votes
4answers
6k views

Escape a ! in the password parameter of wget

I'm trying to execute something like this: wget --user=foo --password=bar! url The ! in the password is causing problems. I've tried escaping it with \, as in --password=bar\! I've tried ...
4
votes
2answers
2k views

How can i make wget download only pages not css images etc?

I want to download an entire website using wget but I don't want wget to download images, videos etc. I tried wget -bqre robots=off -A.html example.com –user-agent=”Mozilla/5.0 (Windows; U; Windows ...
4
votes
3answers
2k views

How to serve a wget --mirror'ed directory of files with questionmarks in them

I'm trying to create a static mirror of a php application (an old php Gallery installation, specifically). The app produces URLs such as: view_album.php?set_albumName=MyAlbum wget downloads these ...
4
votes
7answers
9k views

Wget, Curl, Yum Fail but Ping works - CentOS 5

We currently have three webservers. Servers one and two behave, but I'm having real problems with the third. wget, curl and yum all fail to make their connections - that is, they all hang after ...
3
votes
1answer
1k views

“wget --mirror” Alternative

I'm looking for an alternative to using this, it doesn't have to use FTP, but it should provide the same functionality as the "--mirror" option of wget where it only downloads new and changed files. ...
3
votes
4answers
1k views

Download a file from the shell when button click is required

I have a link to a file that I would like to download from the shell. Unfortunately, the URL redirects to a software license agreement page which requires clicking an accept button. This is fine when ...
3
votes
2answers
11k views

Ubuntu 11.10, using wget/curl fails with ssl

On a completely new install of Ubuntu I'm getting the following errors when using wget: wget https://test.sagepay.com --2012-03-27 12:55:12-- https://test.sagepay.com/ Resolving test.sagepay.com... ...
3
votes
1answer
611 views

Cron Daemon Wget blocked

I've got nginx on Debian 7 without cpanel. I am seting-up my Crontab like this: */45 * * * * wget "http://example.com/cron-url.php" >/dev/null 2>&1 The above cron is being blocked with ...
3
votes
1answer
363 views

Downloading from wget - folder issue

I have a website - http://myradio1.localhost and the directory is currently empty. The virtual host works fine, no issues there. This is my command I'm using to download the site via wget: wget -...
3
votes
2answers
410 views

pgrep wget: what is the details of process id?

when typing pgrep wget, it show process id 10144 but how to know what is the details of this process id
3
votes
1answer
2k views

How can I get a static download link for a Microsoft MSDN Subscriber Download?

I need to be able to download MSDN ISO files directly from power shell, and therefore I am wondering how can I do this? I can view the downloads site at https://msdn.microsoft.com/subscriptions/...
3
votes
2answers
1k views

Curl always returns same 404 page

No matter what URL I specify for curl I always get the same HTML 404 Error page back. If I use the --verbose option, it looks like curl always connects to the same IP address. $ curl --verbose http:/...
3
votes
2answers
4k views

How to exclude certain directories while using wget?

I'd like to download a directory from a FTP, which contains some source codes. Initially, I did this: wget -r ftp://path/to/src Unfortunately, the directory itself is a result of a SVN checkout, so ...
3
votes
2answers
4k views

HTTP downloads stop after some time, resuming is not possible

When I try to download a file via HTTP, the downloads sometimes stop after around 30 MB. The download rates goes down to 0 B/s and no data keeps coming. When I stop the download and resume again, the ...
3
votes
3answers
2k views

Reboot if tomcat7 service is not responding

I am running a web app on a Tomcat server. There is a hard-to-detect problem within the server code that causes it to crash once or twice everyday. I will dig in to correct it when I have time. But ...