GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.
0
votes
4answers
42 views
wget ftp an entire site is very slow
I'm trying to wget an entire site to transfer from Server A to Server B.
It works, but incredibly slowly.
I tested this by uploading a 100mb psd file to my server, and grabbing it with wget ftp. ...
1
vote
3answers
157 views
wget not working with domain on local machine
Basically - I have some PHP scripts that need to be run as cron jobs.
Lets say the script needing to be run is: http://admin.somedomain.com/cron_jobs/get_stats
If I run the script from the local ...
0
votes
3answers
36 views
input wget exclusions from a file?
i'm crawling a forum and I keep stumbling across certain threads that have been going on for ten years.
i can certainly exclude these using wget option:
-X ...
0
votes
3answers
182 views
How can I use wget to get files recursively if the username contains an @?
Firstly my main goal is to try and get FTP files recursively in Centos. However the actual FTP login contains an @ which ruins the syntax. Is there some way to place the username in "double quotes" ...
0
votes
2answers
237 views
WGet from one site on a server to another site on the same server
Hey all, I've recently been asked to administer a couple ubuntu boxes running web servers. I'm a dev by trade so if this question is fairly noob please forgive.
We have about a dozen sites running ...
-1
votes
2answers
405 views
wget downloading index.html instead of tarball?
When I type the following:
http://www.archlinux.org/packages/extra/x86_64/enca/download/
wget is downloading index.html instead of the correct file which is: enca-1.13-2-x86_64.pkg.tar.xz
When I ...
2
votes
1answer
202 views
Reboot if tomcat7 service is not responding
I am running a web app on a Tomcat server. There is a hard-to-detect problem within the server code that causes it to crash once or twice everyday. I will dig in to correct it when I have time. But ...
2
votes
1answer
1k views
using wget through .pac config based proxy server
I want to use wget through a proxy which uses .pac config.
When I googled a bit, I found that .pac is a javascript file and wget cannot parse it based on the following url
...
1
vote
1answer
62 views
SElinux stopping LVS from working with https
LVS/piranha is setup and trying to get it to balance https instead of http. Setup https testing with wget - idea from this link. Works when I do it at the command prompt. With SELinux enforcing, ...
1
vote
1answer
79 views
Serve mirrored (static) web-page with original headers
I have a dynamic webpage which I want to create a "frozen" copy of.
Typically I would do something like wget -m http://example.com, and then put the files in the document root of the web-server.
...
0
votes
1answer
34 views
Why would an HTTP(S) request not be logged in IIS?
I'm troubleshooting a batch file that uses Wget to send requests to a website running in IIS 7. The batch file runs Wget twice, the first time to login to the site via a POST, the second to run some ...
0
votes
1answer
33 views
cannot wget wordpress.org
I am not able to wget wordpress.org from server 63.143.51.2. I am able to wget other sites like google.com
Error I am getting is as follows :
[~]# wget wordpress.org
--2013-04-12 01:42:17-- ...
0
votes
1answer
46 views
What could block the output of a command?
Sometimes I'll attempt a (2>&1) redirection and some/all of the resulting output appears to be silenced.
e.g.
wget -O- http://localhost/test.txt 2>&1
I would expect to see a merge of ...
0
votes
1answer
373 views
Download error for php data using wget
I am trying to download data from my server using wget.
I send the login details and store the cookie. I then rotate through 50 numbers to copy the data into new files. The saved files are always ...
-1
votes
1answer
115 views
Prevent webpages from being downloaded by downloadmanagers with NGINX
I want to create a site that offers themes. People will be able to see the themes and use a demo but I want to prevent cheapskates from using wget or any other download manager to download the demo ...
-1
votes
1answer
161 views
Iptables blocks outgoing HTTP connection
I've setup IPTables with the following script... however, I cannot get outgoing HTTP queries to work (I've tried using lynx & curl).
Background - server is hosted remotely (and I'm connecting via ...
1
vote
0answers
180 views
How to use sitemap.xml to create a static mirror of a CMS
Is there a tool to create a static mirror of a content management system (CMS) that provides a sitemap.xml file?
Ideally, I would point a tool like wget or curl to a sitemap.xml file and have it ...
0
votes
0answers
87 views
why iftop and wget's transfer status varied so much?
the transfer speed is 2147kb/s now
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
29 ...
0
votes
0answers
329 views
wget hangs in http request sent awaiting response in some sites
Using Ubuntu 12.04. wget hangs in http request sent, awaiting response... in some sites. Browser's are not opening sites that are failed in wget. But in WinXP everything works.
This works
...
