Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [wget]

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

13
votes
6answers
5k views

Why the discrepancy between Speedtest and Wget?

My customer complains about low internet speeds. When measured with Speedtest.net speeds are acceptable. Periodic measured downloads are 10% to 30% of the nominal speed. I cannot explain that. Some ...
0
votes
0answers
100 views

Connect to the internet and other containers from inside Docker container

I'm not sure whether to post it on Stackexchange's DevOps website, Network Engineering website, Super User's or here but, I thought this would be the best place for it. In case it's not, please move ...
3
votes
2answers
1k views

Why does curl work with a specific https site, but wget has problems with certificates?

Centos 6. After updating recently ca-certificates I run into some troubles. I have the most recent versions of curl for centos 6: -bash-4.1$ curl -V -v curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7....
0
votes
0answers
38 views

create Linux user to give only wget permission

I own a dedicated server and I'm working on my server along some colleagues. I want to create some users for my colleagues which only be able to use wget command. I mean I don't want to give them ...
0
votes
0answers
43 views

wget issue with ftp download - numbering added and No such file ‘0 myfile.tar’

got strange issue with wget and FTP downlaod (FTP is Windows 2012 R2 so IIS 8.5) when try to FTP download from CentOS wget add Numbers to the listing (or get's it from the index.xml?) but Ubuntu ...
0
votes
0answers
628 views

Cron jobs fail, curl & wget unable to establish SSL handshake

Ubuntu 16.04 PHP 7.0 Apache2 Certbot/LetsEncrypt SSL Cloudflare SSL Strict with Origin Certificate Everything up-to-date. After establishing that SSL wasn't being established, tried the following: ...
2
votes
2answers
79 views

`wget`-ting a website for “local” browsing on a different domain

I need to mirror a website and deploy the copy under a different domain name. The mirroring procedure should be all automatic, so that I can update the copy on a regular basis with cron. The mirror ...
0
votes
2answers
377 views

What does the -c in wget -c Linux command mean?

I am installing Artifactory on an Ubuntu server, and trying to add JFrog repository key. The command is: wget -c -O- "https://bintray.com/user/downloadSubjectPublicKey?username=jfrog" | sudo apt-key ...
0
votes
1answer
195 views

Automated export of phpMyAdmin SQL download with wget

I'm using the following script to attempt to download an SQL export from a phpMyAdmin installation (adapted from this Stackoverflow question) read -p "Username: " USERNAME read -p "Password: " ...
0
votes
0answers
69 views

wget hangs when in AWS user_data

I'm trying to download a file when launching my EC2 instance by passing in a wget command through user_data. However, when I login the wget process is hanging. And if I manually try and download the ...
0
votes
0answers
2k views

How to Enable JavaScript for wget in linux for grabbing a website?

I use wget like this for save site : wget --page-requisites --no-parent --mirror http://example.com/index.html -P /home/ in Some cases It does NOT work, Error is : This site requires Javascript to ...
1
vote
2answers
3k views

Fixing wget certificates

Basically, I was testing different versions of OpenSSL, and somewhere in the process broke wget's path to certificates. wget https://www.google.com Results now in: ERROR: cannot verify www.google....
-3
votes
1answer
398 views

wget download only certain folders from site

I want to download this site with wget, and to do this I'm using this command: wget.exe --no-clobber --no-parent -r --convert-links --page-requisites -U Mozilla "http://www.s-manuals.com/smd/" That'...
0
votes
1answer
1k views

Why can't I get wget to pipe standard output to /dev/null?

When I run this command it works as expected: wget -p https://www.google.com/ After running this command I check the error code with echo $? This returns 0, indicating that last command ran ...
-1
votes
1answer
64 views

stdout and stderr from bash execution to other machine?

Is it possible to send stderr or stdout from a linux machine to ubuntu linux machine using at best wget or other common package at worse ( hopefully not ssh access ) ? So far i can find a way to "...
2
votes
1answer
473 views

wget hangs/freezes when downloading file into NFS

I'm running some experiments with Amazon EFS (General purpose) and EC2 and I have the issue that EFS seems to act unstable. Commands that involve the mounted file system would hang or freeze. For ...
0
votes
1answer
247 views

Using wget to download GlobColour data from hermes ftp server

I want to download the netCDF products from ftp://ftp.hermes.acri.fr. I used wget in cygwin package for Windows systems. My link address for accessing to the data is ftp://ftp.hermes.acri.fr/691969751 ...
0
votes
1answer
331 views

FTP access error in EC2

I have 2 servers with windows and ubuntu running in Amazon EC2. I created the FTP server using plesk in windows and used vsftpd in the ubuntu. I have enabled all TCP ports in both the servers. The ...
1
vote
1answer
768 views

How to download a file and preserver original permissions using wget

How can I download a file that has already executable permissions 755 in another server. I want to: wget https://example.com/pub/register.sh --no-check-certificate and preserver permission register....
4
votes
3answers
2k views

nginx block curl and wget

I have nginx webserver. I have a rich content site and i found that some malicious bots are trying to crawl my content. I blocked any curl or wget coming to my site like this if ($http_user_agent ~* ...
0
votes
0answers
263 views

(mac) Need to automatically backup entire FTP server and create an archive

Is there a way to grab all files from an FTP server using wget (or similar), then create a zip or tar of the downloaded files? I would like to do this with one command or using Automator, and to ...
3
votes
1answer
4k views

Cannot get cURL or wget to validate some SSL certificates

I've noticed that our link checker, which uses cURL, fails more and more often to validate SSL certificates. I'm trying to get to the bottom of this. https://www.bgetem.de/, for instance, opens just ...
-1
votes
1answer
37 views

Stop wget clobbering local file is server unavailable

I have a cron job on a server which once a day uses wget to download "Earth Orientation Parameters" and leapsecond data from a NASA server. Specifically: wget https://gemini.gsfc.nasa.gov/500/oper/...
0
votes
0answers
558 views

Download WordPress site via wget

We have a unique situation whereby we are going to be hosting a website within a company's private network. They have given me RDC access. However, the remote machine has no internet access, so I am ...
2
votes
1answer
489 views

How to set index.html to redirect wget query to an other page/file?

If my server is at example.com, running $ wget example.com will only download the file index.html. How can I make wget either download another file instead of index.html download this file along ...
0
votes
0answers
1k views

How to use wget with redirect and long URL?

I am trying to download the latest release of a web application from GitHub using wget on the shell. As far as I remember, I never had any trouble with wget. I issue the following command: wget '...
1
vote
0answers
538 views

How to set http_proxy env variable and have wget using it transparently before login (Debian)

By setting environment variable http_proxy="http://proxy-server" I'm able to call programs like wget, curl, etc, using the set http proxy transparently. To persist this to the whole system I've added ...
1
vote
0answers
23 views

Start script are not cleared table of processes

I have a proxy server, all ok. But, i have some problems & questions. First: when i make "/etc/init.d/nginx stop" or "/etc/init.d/nginx restart", of my config, it not reload and all nginx process ...
0
votes
0answers
483 views

Wget cannot locally verify issuer authority

I am using my company provided certificate which is self signed. wget https://github.com /This throws the error Other sites work fine. Chrome is able to successfully use this. And i am also able to ...
1
vote
1answer
809 views

Changing all HTML absolute links to relative [closed]

I have a website that has a bunch of absolute addresses, and I need to move it (the whole contents of that website) up a level, so all of the absolute links need to be converted to relative. I know ...
0
votes
3answers
5k views

Running wget as cron job [duplicate]

I want to download backup archives from one server to another. The server has a script that delivers the latest backup archive, the file has specific name for each date. I want wget to preserve that ...
-2
votes
1answer
238 views

SSH/SCP works but not wget

There are two machines: Build: Using jenkins to get code and build war files Deploy: used for deploying the war files to a tomcat server for testing. I am trying to do automatic updates for the war ...
0
votes
1answer
579 views

URL encoding issue with cron and wget

I have a simple bash script: #!/bin/bash wget -q 'http://requestb.in/1fq0r3s1?foo=a%2Bb' -O /dev/null If I run this from the commandline and then inspect at http://requestb.in/1fq0r3s1?inspect I ...
0
votes
1answer
2k views

vshpere wget: bad address 'releases.ubuntu.com'

on a raw vsphere esxi 6 without any vm installed i have to setup vm's and ... remotely so i decided to open ssh to esxi and download an Ubuntu iso using wget this way: wget http://releases.ubuntu....
0
votes
0answers
501 views

Download file from server using a command line

I have 2 different servers, I put my mp3 files in server1, and I am testing to download them from server2 using the wget tool: wget "http://www.example.com/Path-to-mp3-file" I have 2 results: 1) If ...
1
vote
0answers
65 views

Can wget detect remote duplicate files (with different names) and download only one?

I'm retrieving a dynamic website contents recursively. Unfortunately, files are available from different dynamic URLs. For instance, maybe file http://foo.bar/bla.php?q=xyz and http://foo.bar/bla....
1
vote
1answer
640 views

How do I ensure that a CoreOS cloud-config service is able to download files?

I am defining a one-shot service in my CoreOS cloud-config, but it is failing due to not being able to download files from Google Cloud Storage (via wget): Apr 13 11:09:56 staging-node-ys9y.c....
0
votes
2answers
786 views

IPTables: apt-get and wget not working

When resetting IPTables, the apt-get and wget command functions correctly and also downloads what I want. But once I activate this firewall, it isn't functional. Pings still work. I want to allow all ...
0
votes
1answer
4k views

wget giving Segmentation fault

I recently did apt-get update followed by apt-get upgrade on my Debian VPS server. Suddenly (at least) wget stopped working with error Segmentation fault. It's not even trying to do something. So ...
1
vote
1answer
262 views

how can I download web pages from the command line in windows

Well, I'm trying to do some automation using batch file to speed up my daily routine at work, and I need to solve these little questions: I want to install wget command, but using only the pure ...
-3
votes
2answers
120 views

Server not recognizing that wget is installed on OS X Yosemite [closed]

I'm trying to install some software (called Prohits, used for mass spectrometry data management) on a localhost server I set up. One of the requirements for the install is that I have wget installed. ...
1
vote
0answers
159 views

Server socket sitting in FIN_WAIT_1 whilst network doesn't see its traffic

We've been trying to get a grip on a really weird problem where we can wget a page from Apache 2.2.19 on solaris 10 and some permutations of requests reliably take various fixed lengths of time to ...
1
vote
0answers
780 views

why is a wget to http://graph.facebook.com resolving to my server's IP?

This makes absolutely no sense and I spent 3 hours trying to figure this out. Suddenly all my scripts connecting to Facebook stopped working, so I tried to do this: root@s01 [~]# wget http://graph....
4
votes
2answers
11k views

How can I get a static download link for a Microsoft MSDN Subscriber Download?

I need to be able to download MSDN ISO files directly from power shell, and therefore I am wondering how can I do this? I can view the downloads site at https://msdn.microsoft.com/subscriptions/...
0
votes
1answer
25 views

wget blocking subdirectiories

I would like to recursively download a page, but block link like this containing "strony": http://domain.com/rozrywka/strony/2/ my command is wget -r -nd --delete-after -e robots=off -X strony http:...
0
votes
1answer
602 views

Server to Server file transfer through wget

I have a dedicated server in Germany with 120GB data, since I have bought a new dedicated server in U.S, I would like to transfer all the files to new server through FTP connection, to achieve this I'...
2
votes
1answer
6k views

Download an SSL certificate from a remote website through a proxy

I want a script checking my server's certificate. There are very good answers around here how to get this with openssl s_client or gnutls-cli, which works fine.... but NOT if you are behind a proxy! ...
1
vote
1answer
439 views

Which Ubuntu package has the CA for debian.neo4j.org (godaddy)?

The neo4j site shows that you should get their certificate using wget -q -O - http://debian.neo4j.org/neotechnology.gpg.key This, of course, could allow their certificate to be hacked. So, I really ...
-1
votes
1answer
901 views

wget does not work with IPv6 [closed]

My Debian server has both IPV4 and IPV6 addresses. I am using file_get_contents in my php script to get video titles from youtube: if($content=file_get_contents("http://www.youtube.com/...
1
vote
1answer
513 views

APT-GET behind a proxy with Digest Authentication

I'd like to use apt-get (and other ubuntu tools) to download software and keep it updated. Unfortunately, my company has set a squid proxy to accept digest authentication only. I've seen somewhere (...