GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

learn more… | top users | synonyms

0
votes
0answers
4 views

Why is cURL slower than wget [migrated]

Test page: https://www.beobank.be/nl/Home.aspx This is the result using wget (or a real browser): # time wget https://www.beobank.be/nl/Home.aspx -O /dev/null --2015-01-26 12:05:46-- ...
-2
votes
2answers
2k views

What does -q and -O mean?

What does -q and -O mean in the following cronjob? */30 * * * * /usr/bin/wget -q -O /dev/null http://example.com/moodle/admin/cron.php Thanks in advance.
0
votes
1answer
39 views

how to force wget to ignore certain redirects

I am trying to wget a list of urls (images) some of them no longer exist and the host redirect to a generic "this image doesn't exist" page, of which I know the url of. I would like to wget the file ...
2
votes
1answer
1k views

curl and wget attempting an ipv6 connection

On my dedicated server (hosted by OVH), that is running a fresh install of Ubuntu 14.04, curl and wget take approximately 10 seconds to complete a simple request. $ curl -v google.com * Rebuilt URL ...
1
vote
1answer
48 views

Use tmux for managing multiple downloaders as a supervisorctl service?

I have many data-servers I need to download data from via http as soon as it is available. For each server I start a bash "while true"-loop and within that a wget to poll the server for new data. To ...
-1
votes
1answer
42 views

Downloading ftp with wget when the username is an email address [duplicate]

I want to copy all of the files and folders from one host to another. The files are on the old host site at /var/www/html. I only have FTP access to that server, and I can't TAR all the files. I ...
2
votes
3answers
1k views

How to serve a wget --mirror'ed directory of files with questionmarks in them

I'm trying to create a static mirror of a php application (an old php Gallery installation, specifically). The app produces URLs such as: view_album.php?set_albumName=MyAlbum wget downloads these ...
3
votes
1answer
136 views

Cron Daemon Wget blocked

I've got Nginx on Debian 7 without cpanel. I am seting-up my Crontab like this: */45 * * * * wget "http://example.com/cron-url.php" >/dev/null 2>&1 The above cron is being blocked with ...
5
votes
4answers
17k views

Is there a built-in command-line tool under Windows like wget/curl?

I come from a Linux/Unix background and I have been wondering if Windows has a binary that can download files from the console. I would like to automate a certain process and one of my requirements ...
0
votes
1answer
197 views

Ubuntu SSL connection with TLS connection error [closed]

I am using Ubuntu 12.04, installed at VirtualBox (Mac) using vagrant. I am not using any proxy server. In my company network, when I try curl -1vsS ...
-1
votes
1answer
56 views

Error when using wget to https via proxy

My browser uses my PC's proxy setting. As far as I can see it uses some $http_URL. So, I put https_proxy=$http_URL http_proxy=$http_URL The http connection work fine wget $http_URL but I can't ...
2
votes
1answer
306 views

ShellShock test shows wget and curl access

I've fixed shellshock bug on my Debian 6 server and while testing on http://shellshock.brandonpotter.com/ I get "No Vulnerabilities Found" and that's OK but they also check other things and in the ...
0
votes
2answers
103 views

Why does downloading a large file with wget seem to consume nearly all available RAM? [closed]

I am trying to download a file of 580MB with the following simple wget command: wget http://example.com/file.ext The server has 16 GB installed RAM of and during the entire download the memory ...
2
votes
4answers
5k views

curl succeeds on an https request, wget does not

When I run curl from the command line curl "https://example.com" It succeeds immediately, returning the result of the request. When I run the same wget command wget https://example.com ...
3
votes
2answers
2k views

using wget through .pac config based proxy server

I want to use wget through a proxy which uses .pac config. When I googled a bit, I found that .pac is a javascript file and wget cannot parse it based on the following url ...
2
votes
6answers
33k views

WGET can't resolve host

I've got 2 machines on the same sub-net running Debian 5.0. They use the same DNS (/etc/resolv.conf), they route to the same gateway (#route) and they have the same iptables settings (#iptables -L). ...
0
votes
1answer
40 views

Downloading a file over HTTP while it is being downloaded on the server

I have a download going on on a internal network over HTTP (only available protocol) using wget. The content of interest is NOT reachable from the WWW. The file is being downloaded to a public http ...
1
vote
3answers
6k views

why i cannot download jdk from oracle web site directly without AuthParam?

that is download with the following command, why it fails to download that file? wget http://download.oracle.com/otn-pub/java/jdk/6u35-b10/jdk-6u35-linux-i586.bin the following command works, but ...
15
votes
5answers
3k views

Piping wget output to /dev/null in cron

I am running the following command every 5 minutes in my crontab to keep Phusion Passenger alive. */5 * * * * wget mysite.com > /dev/null 2>&1 When I run this it performs a wget on the ...
0
votes
4answers
2k views

wget doesn't take variable http_proxy

If I run: echo $http_proxy It output: http://user:pass@host:port But when I run: wget www.google.com It output: Resolviendo www.google.com... 209.85.195.104 Caching www.google.com => ...
-5
votes
2answers
56 views

Bash Script that downloads a file and checks the value if 1 or 0 [closed]

I need a Bash Script for Debian which downloads a file, that file containts 1 or 0, 1 when the services is running, 0 when not. So how can I check if the file contains 1 or 0?
1
vote
1answer
110 views

Under what scenarios would Apache allow a mismatched HTTP_HOST to pass through?

Someone has managed to pass an undefined HTTP_HOST server variable to my application script, triggering a series of errors. I am quite perturbed but am unable to replicate this behaviour. My httpd ...
1
vote
3answers
2k views

wget is working only when used with sudo

I'm having quite a strange behavior with wget since yesterday. I can download files by using sudo wget, but when I try the same file with only wget, I can get this error: yusufh@ubuntu-yuh:~$ wget ...
0
votes
2answers
256 views

“wget ca-directory=/etc/ssl/certs” failing on Ubuntu 14.04

Ubuntu 14.04's default wget cannot handle SSL. How can I fix this? I do not want to use --no-certificate-check. The certificate I want to authenticate is authenticated against the Root CA from ...
6
votes
3answers
3k views

wget starts downloading then stops “cannot write to”

I'm using wget to mirror some files across from one server to another. I'm using the following command: wget -x -N -i http://domain.com/filelist.txt -x = Because I want to keep the directory ...
67
votes
4answers
69k views

how to download the ssl certificate from a website?

I want to download the ssl certificate from, say https://www.google.com, using wget or any other commands. Any unix command line? wget or openssl?
0
votes
1answer
188 views

Why does my cron job sometimes fail on wget: unable to resolve host address [closed]

My cron job runinng properly .But sometimes it get fail with following error message" failed: No address associated with hostname. wget: unable to resolve host address "
0
votes
1answer
48 views

User WinWget to keep web site alive in a Windows Server 2003

I have a site that must stay alive due to a service that runs and check a directory for changes. The site is running in IIS at a Windows Server 2003 and the solution I came up it that I will ...
1
vote
2answers
454 views

Cron getting timeout when attempting to access url (wget).

I'm sure this is simple and I've been digging but no answer is quite as specific as I need it to be. The goal is easy, have Cron hit a url on my server every 5 minutes. All of that is set up and ...
1
vote
3answers
4k views

Execute curl requests in parallel in bash

What is the best way to execute 5 curl requests in parallel from a bash script? I can't run them in serial for performance reasons.
2
votes
3answers
1k views

Reboot if tomcat7 service is not responding

I am running a web app on a Tomcat server. There is a hard-to-detect problem within the server code that causes it to crash once or twice everyday. I will dig in to correct it when I have time. But ...
1
vote
2answers
342 views

Allow connection from localhost to the HTTPS/Port 443 Protocol

We've recently set up a new development server and have been experiencing problems when trying to connect to an API via an instance of SoapClient. I think I've narrowed down the issue to being the ...
1
vote
1answer
94 views

Wget and recursive downloads

A directory listings enabled folder has 2 zip files. I am trying to download only one of them (the latest). Problem is the zip file name is a random string. The command I have so far is wget -r -nH ...
2
votes
3answers
295 views

wget and port forwarding

I have a weird problem. I can run this command just fine from various servers: wget --debug '--http-user=USER123' '--http-passwd=PASSWORD' http://GW-BOX:9091/weijhkdsvn/v9_odbc//CRONTAB.2014020 ...
0
votes
1answer
987 views

wget ftp connection failing after PASV command

In attempting to transfer all files from one web server ("source") to another ("destination"), the wget command is connecting via FTP, but cannnot proceed beyond the PASV command. I'm using an SSH ...
1
vote
1answer
200 views

Ghost running on docker container does not respond to http requests from other container

I am setting up a ghost blog instance inside a docker container. I bound the ghost server to host 0.0.0.0 and exposed the port on which it's listening (port 2368). If I run wget to the ghost ...
5
votes
3answers
23k views

Why wget doesn't verify SSL certificates?

I have a problem with my Fedora 8 installation. It looks that wget doesn't know how to verify SSL certificates any more. It's strange because I have another Fedora 8 box which I believe has the same ...
0
votes
1answer
77 views

Protect public directories from download via wget and curl

My users upload their profile images and I save them in a public folder. The file names are not predictable (they are based on md5 hash of user id with salt). When someone views the user's profile, ...
0
votes
1answer
1k views

Wget on busybox v1.9.1

I have busybox 1.9.1 runing in huawei router. There is few commands, one of them wget and I want to use it, but there is some strange syntax. When I type wget www.google.com like I do normaly on my ...
5
votes
6answers
6k views

How to download with wget without following links with parameters

I'm trying to download two sites for inclusion on a CD: http://boinc.berkeley.edu/trac/wiki http://www.boinc-wiki.info The problem I'm having is that these are both wikis. So when downloading with ...
1
vote
1answer
164 views

Command line tools to monitor POST/GET requests of a loaded website

When I load a website with Firefox and open up the Network Profiler, I can see all the POST and GET requests the site is making. I would like this functionality from the command line. Does any one ...
38
votes
8answers
31k views

Making `wget` not save the page

I'm using the wget program, but I want it not to save the html file I'm downloading. I want it to be discarded after it is received. How do I do that?
1
vote
1answer
157 views

Why does wget from ftp-sites sometimes create a few empty files when it downloads the rest of the files without problems?

I have downloaded 80.000+ image files stored on a FTP-site using wget(1) and most of the files where downloaded without problems but when I did a small check (using file(1)) I just realized that a few ...
0
votes
3answers
2k views

download only new files from ftp using WGET/cUrl

i have Windows 2008 Server on which i store DB backups on daily basis. I want to be able to download new files only using wget, curl or windows built-in FTP doesn't matter. Can you help me with ...
-1
votes
2answers
2k views

Installing make with wget

How can I install make with wget on CentOS? I tried: cd /tmp wget ftp.gnu.org/pub/gnu/make/make-3.81.tar.gz tar xfz make-3.81.tar.gz cd make-3.81 PATH=/usr/local/bin:/usr/bin:/bin ./configure ...
0
votes
2answers
466 views

“Issued certificate not yet valid.” with wget?

I am getting the following error when using wget to fetch a file: Issued certificate not yet valid. To connect to www.atlassian.com insecurely, use ‘--no-check-certificate’. Testing with openssl ...
0
votes
1answer
211 views

Unable to get cron to append output from wget

I have a cronjob setup and running, but the logfile keeps getting overwritten every time it runs. I would like for the wget output to be appended each time the cron runs. Why is this and how do I fix ...
0
votes
2answers
716 views

Not able to wget to ftp server

I am trying to wget a ftp server from a remote machine. The command is not getting past 'Logging in as anonymous'. This is what i am getting. wget ...
0
votes
6answers
848 views

is there a problem in disabling curl and wget?

Is it recommended to disable curl and wget to disallow anyone from accessing my websites using those commands? Or is it needed for some search engines to access my websites? What are your ...
1
vote
3answers
16k views

How to install wget on this?

I did download RubyStack 2.0.3 for VMWare (Ubuntu 9.10) but I cannot download anything on it! It appears that all basic utilities are missing/screwed: bitnami@linux:/var/tmp$ wget -bash: wget: ...