GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

learn more… | top users | synonyms

-1
votes
2answers
951 views

wget downloading index.html instead of tarball?

When I type the following: http://www.archlinux.org/packages/extra/x86_64/enca/download/ wget is downloading index.html instead of the correct file which is: enca-1.13-2-x86_64.pkg.tar.xz When I ...
0
votes
1answer
2k views

Wget - download all links from a http location (not recursivly)

I have a link to an http page that has a structure like this: Parent Directory - [DIR] _OLD/ 01-Feb-2012 06:05 - [DIR] _Jan/ ...
1
vote
3answers
184 views

Limiting specific process to just one per user

My scripts are using wget to retrieve data from Internet. When many users are using this script I got very high load (about 20.00), because of disk I/O. Wget is automatically started each hour by ...
0
votes
2answers
1k views

FTP file download using Wget

I am using the wget command for download FTP files, when i download the FTP file its showing error "Event not fount". Here i use the password like some below charater ! so its showing this error ...
2
votes
2answers
1k views

How can i make wget download only pages not css images etc?

I want to download an entire website using wget but I don't want wget to download images, videos etc. I tried wget -bqre robots=off -A.html example.com –user-agent=”Mozilla/5.0 (Windows; U; Windows ...
3
votes
1answer
313 views

Downloading from wget - folder issue

I have a website - http://myradio1.localhost and the directory is currently empty. The virtual host works fine, no issues there. This is my command I'm using to download the site via wget: wget ...
1
vote
1answer
3k views

Wget from HTTPS url using user/password -still getting 400 error

I'm trying to run Wget for a certain HTTP url, also using http-user and http-password command line options (tried the --user and --password, same results). Using the --no-check-certificate options as ...
3
votes
3answers
1k views

How to serve a wget --mirror'ed directory of files with questionmarks in them

I'm trying to create a static mirror of a php application (an old php Gallery installation, specifically). The app produces URLs such as: view_album.php?set_albumName=MyAlbum wget downloads these ...
-1
votes
1answer
948 views

Using wget to get count of pages below a link?

I've been using a sitemapping tool to get a simple count of links below a specific url. The free trial has ended, so I figure that rather than paying $70 for what is very simple functionality, I ...
1
vote
2answers
743 views

Can someone explain to me what “wget -O - -q icanhazip.com” mean?

I've used this command to find the IP of the server. Can someone explain what the command means? I want to learn. So far I know: "wget" is a free utility for non-interactive download of files from ...
0
votes
1answer
75 views

How to download files that are dynamically served

I've run into this a few times when a site serves a variable file i try to download wget ...
2
votes
2answers
173 views

how can i use apache log files to recreate usage scenario

Recently i installed a website that had too many requests and it was too slow. Many improvements have been made to the web site code and we've also bought a new server. I want to test the new server ...
3
votes
1answer
2k views

How to exclude certain directories while using wget?

I'd like to download a directory from a FTP, which contains some source codes. Initially, I did this: wget -r ftp://path/to/src Unfortunately, the directory itself is a result of a SVN checkout, so ...
0
votes
2answers
962 views

host resolution extremely slow using wget on local dev machine. fast using browser

On my local Ubuntu 10.04 dev machine, if I do a wget for a web address such as "wget http://www.google.com", it gets stuck on "Resolving www.google.com" for up to 30 seconds before the response is ...
0
votes
1answer
340 views

Wget not working

wget is not working properly on my debian server. if I wget http://www.google.com, it says it cannot resolve the host, when I ping google.com, I receive a successful pong and when I wget ...
7
votes
3answers
29k views

Why wget doesn't verify SSL certificates?

I have a problem with my Fedora 8 installation. It looks that wget doesn't know how to verify SSL certificates any more. It's strange because I have another Fedora 8 box which I believe has the same ...
6
votes
2answers
6k views

How do I pipe the output of uptime/df to curl?

I want to send the output of uptime and df commands to a web app using cURL. I tried: uptime | curl http://someurl.com -T - But that didn't seem to send the data. What is the proper way to send ...
3
votes
2answers
994 views

Curl always returns same 404 page

No matter what URL I specify for curl I always get the same HTML 404 Error page back. If I use the --verbose option, it looks like curl always connects to the same IP address. $ curl --verbose ...
0
votes
1answer
798 views

wget and connection errors / timeouts

I was using wget the in the last week to recursively download a whole website of html pages. I used it this way: wget --recursive --no-clobber --page-requisites --html-extension --convert-links ...
-3
votes
3answers
2k views

Automatic download a site with *wget*: how to prevent? [closed]

How to prevent automatic download with wget for a website? Could you give links to articles please? Thanks.
4
votes
2answers
2k views

what is the default rate unit in wget?

When I use wget to download a large file, it gives me "56M/s". I want to know if it means 56 megabits per second or 56 megabytes per second. Thanks! Like this: 231,997,440 61.4M/s in 4.0s
-1
votes
1answer
1k views

wget --timeout doesn't time out an active connection

When using --timeout with wget, I was expecting that if the transaction hasn't completed within the timeout period, then it would abort. However, it seems to mean (with respect to read-timeout) that ...
2
votes
4answers
6k views

Why is wget hanging?

wget to a single specific url from one of my servers keeps getting timeouts. All other urls from this box work fine. This url works OK from any other boxes I have. Here's the output: wget -T 10 ...
3
votes
1answer
2k views

Can wget convert links within a local html document?

I know that wget can fetch a remote page and its dependencies and rewrite the html so that image src attributes reference the newly downloaded images. I am trying to convert local html files that ...
0
votes
3answers
3k views

download only new files from ftp using WGET/cUrl

i have Windows 2008 Server on which i store DB backups on daily basis. I want to be able to download new files only using wget, curl or windows built-in FTP doesn't matter. Can you help me with ...
0
votes
1answer
316 views

Am I doing ubuntu 10.0.4 software installation correctly (packages and none-packages)?

Background: I first installed everything using a terminal and "sudo su", i.e. as root. Then I installed jenkins, and then the jenkins user could not run any android sdk tools in /root/androidsdk/tools ...
1
vote
2answers
494 views

Download file in Linux when file location is unknown

Sorry if this is unclear, but I'm trying to set up a script that downloads a file. Currently, my method of downloading the file is by clicking on a link like so: ...
15
votes
3answers
3k views

Wget: Is there a way to pass username and password from a file?

Is there a way to pass the username and password from a file instead of the command line via --user and --password? Background: I want to run wget via cron and don't want the username/password show ...
1
vote
1answer
513 views

Translate web page from shell

I'm trying to do translate a web page automatically with a shell script. Here is the interesting part: wget http://www.google.com/translate?langpair=en%7Ces&u=http%3A%2F%2Fasdf.com I'm getting ...
2
votes
3answers
409 views

wget application replacement

Is there any other application similar to wget? If yes can you post a link?
1
vote
4answers
992 views

wget an remote website index directory

Is it possible to wget an entire index/parent directory with wget? Because I am trying to transfer images to a VPS of mine from my old shared host. When I say an index/parent directory, I meant like ...
1
vote
2answers
128 views

wget has different response times for same fetch

Any ideas why one fetch would fetch a file (big picture in this example) fast while another fetch would fetch same file slow? Not seeing this issue at home. Not seeing this issue through ip ...
0
votes
2answers
4k views

Using wget to save sequential files as well as renaming the file extension

I run a cron job that requests a snapshot from a remote webcam at a local address: wget http://user:pass@10.0.0.50/snapshot.cgi This creates the files snapshot.cgi, snapshot.cgi.1, snapshot.cgi.2, ...
1
vote
4answers
251 views

Internet speed in Linux via command

I am running several sites and I was trying wget and I noticed slow speed. So I suspect that my sites are taking considerable amount of traffic. Question is, how I can really detect the total ...
1
vote
3answers
5k views

wget: download a website which is password protected using a form

I have a problem with downloading a website using wget. First I need to login to the website with a username and password: wget --save-cookies cookies.txt --post-data --cookies=on ...
3
votes
5answers
4k views

Multithreaded downloading with shell script

Let's say I have a file with lots of URLs and I want to download them in parallel using arbitrary number of processes. How can I do it with bash?
4
votes
2answers
10k views

Alternatives to CURL, WGET

Is anyone aware of any alternatives to curl and wget? The key functional requirement I'm looking for is to be able to execute an HTTPS GET request against a known URL. The reason I'm looking for an ...
0
votes
3answers
358 views

How can I use wget to get files recursively if the username contains an @?

Firstly my main goal is to try and get FTP files recursively in Centos. However the actual FTP login contains an @ which ruins the syntax. Is there some way to place the username in "double quotes" ...
2
votes
2answers
1k views

Why is this bash command not echoing into a variable and what can I do to improve?

I've got a bash script I'm working to improve, and have put on a great fix thanks to Dennis Williamson. Unfortunately, one of the lines no longer echoes into a variable I can manipulate, rather dumps ...
2
votes
1answer
242 views

Why does the time it takes to run this script differ so much between user and system and how I might correct the discrepancy?

My application sits behind a load balancer, and every once in a while I like to do a status check on each machine to get an idea of the time it takes to return an index.html document on each machine. ...
2
votes
2answers
3k views

WGet or cURL: Mirror Site from http://site.com And No Internal Access

I have tried wget -m wget -r and a whole bunch of variations. I am getting some of the images on http://site.com, one of the scripts, and none of the CSS, even with the fscking -p parameter. The ...
-2
votes
1answer
312 views

cpanel cron job not working [closed]

I have created a cron job in cpanel. The result is this: * * * * * /usr/local/bin/wget http://example.com/facturi-restante/cron The problem is that is not working. Can anyone give me a ...
12
votes
3answers
5k views

Why is only one wget command working in my crontab?

I wish to fetch content from a PHP script on my server two times a day, altering a query variable lang to set what language we want, and save this content in two language specific files. This is my ...
3
votes
2answers
3k views

using wget through .pac config based proxy server

I want to use wget through a proxy which uses .pac config. When I googled a bit, I found that .pac is a javascript file and wget cannot parse it based on the following url ...
8
votes
4answers
10k views

Wget a script and run it

I want to download a script from: http://dl.dropbox.com/u/11210438/flockonus-stack.sh and execute it. My guess is, to use wget renaming it, chmod it, and execute. What are the commands for doing ...
5
votes
1answer
730 views

What to do if wget's progress:mega doen't cut it anymore?

I have script for downloading large files in a queue. Now I would really appreciate it, if I had a decent log. Specifying the -o or -a option is apparently specially made for that, however as of the ...
1
vote
1answer
968 views

wget not getting all files

I have issues using wget, basically I use it like: wget -m ftp://username:password@hostname.com:/path/to/dirs After everything's complete (it says xxx files in xx MB for xx seconds), however, when ...
0
votes
3answers
433 views

Download Sun Studio via CLI

Can anybody please guide me how to download the sun studio from CLI. I was using wget and lynx programs but not worked. As I have only SSH access to my server and I cant not download it on local ...
0
votes
3answers
192 views

transfer code from one server to other server

I wanted to transfer new code into my new production server. I have code files on my development server. Instead of uploading files using FTP from my local machine, there is other way to transfer ...
3
votes
2answers
378 views

pgrep wget: what is the details of process id?

when typing pgrep wget, it show process id 10144 but how to know what is the details of this process id