GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

learn more… | top users | synonyms

3
votes
2answers
749 views

Curl always returns same 404 page

No matter what URL I specify for curl I always get the same HTML 404 Error page back. If I use the --verbose option, it looks like curl always connects to the same IP address. $ curl --verbose ...
0
votes
1answer
591 views

wget and connection errors / timeouts

I was using wget the in the last week to recursively download a whole website of html pages. I used it this way: wget --recursive --no-clobber --page-requisites --html-extension --convert-links ...
-3
votes
3answers
1k views

Automatic download a site with *wget*: how to prevent? [closed]

How to prevent automatic download with wget for a website? Could you give links to articles please? Thanks.
2
votes
2answers
2k views

what is the default rate unit in wget?

When I use wget to download a large file, it gives me "56M/s". I want to know if it means 56 megabits per second or 56 megabytes per second. Thanks! Like this: 231,997,440 61.4M/s in 4.0s
-1
votes
1answer
906 views

wget --timeout doesn't time out an active connection

When using --timeout with wget, I was expecting that if the transaction hasn't completed within the timeout period, then it would abort. However, it seems to mean (with respect to read-timeout) that ...
2
votes
4answers
4k views

Why is wget hanging?

wget to a single specific url from one of my servers keeps getting timeouts. All other urls from this box work fine. This url works OK from any other boxes I have. Here's the output: wget -T 10 ...
3
votes
1answer
1k views

Can wget convert links within a local html document?

I know that wget can fetch a remote page and its dependencies and rewrite the html so that image src attributes reference the newly downloaded images. I am trying to convert local html files that ...
0
votes
3answers
2k views

download only new files from ftp using WGET/cUrl

i have Windows 2008 Server on which i store DB backups on daily basis. I want to be able to download new files only using wget, curl or windows built-in FTP doesn't matter. Can you help me with ...
0
votes
1answer
275 views

Am I doing ubuntu 10.0.4 software installation correctly (packages and none-packages)?

Background: I first installed everything using a terminal and "sudo su", i.e. as root. Then I installed jenkins, and then the jenkins user could not run any android sdk tools in /root/androidsdk/tools ...
1
vote
2answers
412 views

Download file in Linux when file location is unknown

Sorry if this is unclear, but I'm trying to set up a script that downloads a file. Currently, my method of downloading the file is by clicking on a link like so: ...
14
votes
2answers
2k views

Wget: Is there a way to pass username and password from a file?

Is there a way to pass the username and password from a file instead of the command line via --user and --password? Background: I want to run wget via cron and don't want the username/password show ...
1
vote
1answer
435 views

Translate web page from shell

I'm trying to do translate a web page automatically with a shell script. Here is the interesting part: wget http://www.google.com/translate?langpair=en%7Ces&u=http%3A%2F%2Fasdf.com I'm getting ...
2
votes
3answers
354 views

wget application replacement

Is there any other application similar to wget? If yes can you post a link?
1
vote
4answers
735 views

wget an remote website index directory

Is it possible to wget an entire index/parent directory with wget? Because I am trying to transfer images to a VPS of mine from my old shared host. When I say an index/parent directory, I meant like ...
1
vote
2answers
125 views

wget has different response times for same fetch

Any ideas why one fetch would fetch a file (big picture in this example) fast while another fetch would fetch same file slow? Not seeing this issue at home. Not seeing this issue through ip ...
0
votes
2answers
3k views

Using wget to save sequential files as well as renaming the file extension

I run a cron job that requests a snapshot from a remote webcam at a local address: wget http://user:pass@10.0.0.50/snapshot.cgi This creates the files snapshot.cgi, snapshot.cgi.1, snapshot.cgi.2, ...
1
vote
4answers
243 views

Internet speed in Linux via command

I am running several sites and I was trying wget and I noticed slow speed. So I suspect that my sites are taking considerable amount of traffic. Question is, how I can really detect the total ...
1
vote
3answers
4k views

wget: download a website which is password protected using a form

I have a problem with downloading a website using wget. First I need to login to the website with a username and password: wget --save-cookies cookies.txt --post-data --cookies=on ...
2
votes
5answers
3k views

Multithreaded downloading with shell script

Let's say I have a file with lots of URLs and I want to download them in parallel using arbitrary number of processes. How can I do it with bash?
2
votes
2answers
7k views

Alternatives to CURL, WGET

Is anyone aware of any alternatives to curl and wget? The key functional requirement I'm looking for is to be able to execute an HTTPS GET request against a known URL. The reason I'm looking for an ...
0
votes
3answers
339 views

How can I use wget to get files recursively if the username contains an @?

Firstly my main goal is to try and get FTP files recursively in Centos. However the actual FTP login contains an @ which ruins the syntax. Is there some way to place the username in "double quotes" ...
2
votes
2answers
1k views

Why is this bash command not echoing into a variable and what can I do to improve?

I've got a bash script I'm working to improve, and have put on a great fix thanks to Dennis Williamson. Unfortunately, one of the lines no longer echoes into a variable I can manipulate, rather dumps ...
2
votes
1answer
233 views

Why does the time it takes to run this script differ so much between user and system and how I might correct the discrepancy?

My application sits behind a load balancer, and every once in a while I like to do a status check on each machine to get an idea of the time it takes to return an index.html document on each machine. ...
2
votes
2answers
2k views

WGet or cURL: Mirror Site from http://site.com And No Internal Access

I have tried wget -m wget -r and a whole bunch of variations. I am getting some of the images on http://site.com, one of the scripts, and none of the CSS, even with the fscking -p parameter. The ...
-1
votes
1answer
299 views

cpanel cron job not working

I have created a cron job in cpanel. The result is this: * * * * * /usr/local/bin/wget http://example.com/facturi-restante/cron The problem is that is not working. Can anyone give me a ...
7
votes
3answers
4k views

Why is only one wget command working in my crontab?

I wish to fetch content from a PHP script on my server two times a day, altering a query variable lang to set what language we want, and save this content in two language specific files. This is my ...
3
votes
2answers
2k views

using wget through .pac config based proxy server

I want to use wget through a proxy which uses .pac config. When I googled a bit, I found that .pac is a javascript file and wget cannot parse it based on the following url ...
7
votes
4answers
6k views

Wget a script and run it

I want to download a script from: http://dl.dropbox.com/u/11210438/flockonus-stack.sh and execute it. My guess is, to use wget renaming it, chmod it, and execute. What are the commands for doing ...
5
votes
1answer
591 views

What to do if wget's progress:mega doen't cut it anymore?

I have script for downloading large files in a queue. Now I would really appreciate it, if I had a decent log. Specifying the -o or -a option is apparently specially made for that, however as of the ...
1
vote
1answer
740 views

wget not getting all files

I have issues using wget, basically I use it like: wget -m ftp://username:password@hostname.com:/path/to/dirs After everything's complete (it says xxx files in xx MB for xx seconds), however, when ...
0
votes
3answers
398 views

Download Sun Studio via CLI

Can anybody please guide me how to download the sun studio from CLI. I was using wget and lynx programs but not worked. As I have only SSH access to my server and I cant not download it on local ...
0
votes
3answers
178 views

transfer code from one server to other server

I wanted to transfer new code into my new production server. I have code files on my development server. Instead of uploading files using FTP from my local machine, there is other way to transfer ...
3
votes
2answers
347 views

pgrep wget: what is the details of process id?

when typing pgrep wget, it show process id 10144 but how to know what is the details of this process id
0
votes
2answers
387 views

DNS issue on Fedora 12? wget wordpress.org fails where wget www.google.com works

I'm administering a Fedora 12 box, but am quite new to networking specifics. Recently one of our WordPress apps hosted on our server has stopped being able to perform its auto-update or auto-download ...
1
vote
3answers
2k views

wget is working only when used with sudo

I'm having quite a strange behavior with wget since yesterday. I can download files by using sudo wget, but when I try the same file with only wget, I can get this error: yusufh@ubuntu-yuh:~$ wget ...
0
votes
1answer
420 views

Download error for php data using wget

I am trying to download data from my server using wget. I send the login details and store the cookie. I then rotate through 50 numbers to copy the data into new files. The saved files are always ...
2
votes
2answers
1k views

Retrieve FTP directory tree from the command line

What would be the fastest way to recursively retrieve entire directory listing from an ftp server using wget/curl/whatever? I don't need to download any files, just directory and file names. Basically ...
1
vote
1answer
2k views

Using lynx or wget in cron to call a php script?

I'm examining a particular setup, and they have their crontab as follows: 0 * * * * lynx http://www.example.com/cron/scriptA.php Of course, this relies on the 'security-by-obscurity' precept, ...
0
votes
2answers
294 views

WGet from one site on a server to another site on the same server

Hey all, I've recently been asked to administer a couple ubuntu boxes running web servers. I'm a dev by trade so if this question is fairly noob please forgive. We have about a dozen sites running ...
0
votes
2answers
40 views

URL arguments and Webserver configuration

I am trying to archive an old wordpress blog of mine. So far the best way I have found to do this is via wget. The problem is the name of the files it generates, like ...
0
votes
6answers
821 views

is there a problem in disabling curl and wget?

Is it recommended to disable curl and wget to disallow anyone from accessing my websites using those commands? Or is it needed for some search engines to access my websites? What are your ...
0
votes
2answers
116 views

I'm seeing a lot of wget requests, should I be worried

I've got a web based application that I've developed and I'm seeing a lot of wget requests on the site. Should I be worried that someone is trying to hack/rip the site. I don't know what they would ...
0
votes
2answers
3k views

How to tell wget to use the name of the target file behind the HTTP redirect?

Let's a consider link such as htttp://www.mysite.net/download.php?id=267 that redirects to htttp://www.mysite.net/downloads/myfile.tar.gz If I execute 'wget ...
1
vote
3answers
364 views

File timestamp not changing even with touch

I have the following problem. I have a bash script that uses wget to get a few files from the build server, and then scp's them over to the production system: Here is the relevant snippet: #Several ...
2
votes
7answers
4k views

Wget, Curl, Yum Fail but Ping works - CentOS 5

We currently have three webservers. Servers one and two behave, but I'm having real problems with the third. wget, curl and yum all fail to make their connections - that is, they all hang after ...
2
votes
1answer
253 views

How to prevent external website from being downloaded in wget?

I'm using wget to download useful website recursively: wget -k -m -r -q -t 1 http://www.web.com/ But external website also downloaded along with the website I want. How to prevent this external ...
1
vote
0answers
241 views

How to use sitemap.xml to create a static mirror of a CMS

Is there a tool to create a static mirror of a content management system (CMS) that provides a sitemap.xml file? Ideally, I would point a tool like wget or curl to a sitemap.xml file and have it ...
10
votes
2answers
15k views

Where can I (safely) get libintl3.dll?

I am having problems running the Windows version of wget on a Windows 7 machine. When I run the command I get this error: The program can't start because libintl3.dll is missing from your ...
0
votes
1answer
1k views

wget - specify directory and rename the file

I'm trying to download multiple files and need to rename as I download, how can I do that and specify the directory I want them to download to? I know i need to be using -P and -O to do this but it ...
1
vote
1answer
134 views

wget only works for some users

Currently have a strange situation: wget only works for one user on a system -- and root isn't it. There are three user accounts, user A, B and root. User A can use wget with no problems. User B and ...