Questions tagged [wget]

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

Filter by
Sorted by
Tagged with
1
vote
0answers
59 views

How to add correct chain certificate when acessing url with wget? [migrated]

There is a site (https://datashare.is.ed.ac.uk/bitstream/handle/10283/3192/CINIC-10.tar.gz) which I know is misconfigured and I can get the new chained certificate from here: https://whatsmychaincert....
0
votes
1answer
32 views

wget -m delete local files that are not existing anymore on the server

I use wget -m to mirror my webspace to my nas. So far it's working great, but I noticed that when a file on the server got deleted, it will be still there locally. Is there any way to delete the local ...
0
votes
0answers
68 views

Can I make curl (or wget) ignore a specific TLS error? (bypassing expired certificate without disabling cert validation)

I'm working on a script which depends on a remote API endpoint which I do not control. Today, my script stopped working, because the endpoint's SSL certificate expired today and they haven't yet fixed ...
0
votes
0answers
47 views

Wget- Skip hosts that are down

I'm using wget to bulk download a website, and it grabs files from other servers, however, some hosts are down. All wget does is just hang there, so I'd like it to just skip these websites. Is there ...
1
vote
0answers
20 views

Is there a tool I can use to test out multiple vpn tunnels in one machine?

I use wget -q -O https://checkmyip.com/csv to determine if I am connected to a VPN but I want to verify multiple VPN tunnels on the same machine. Is there a tool that can allow me to send a command ...
2
votes
2answers
42 views

Can't serve static files with ampersand in name

I have a website mirrored with wget and I'd like to serve it via Apache. The problem is that I can't access files with special chars. For example, I have files with names like this: $ ls -N components/...
0
votes
0answers
27 views

Nginx does not cache wget requests

I'm using Nginx as a reverse proxy for caching. The caching is working but when I use wget command the files are downloaded on the cache folder but I cannot see them in the access log. To solve this ...
0
votes
1answer
38 views

wget website recusively from localhost without using bandwith

i'm looking to download recusively my wordpress website into static using wget, the problem is that whenever i do that, it's using too much bandwith (3.5gb) even though i end up downloading 20mb which ...
0
votes
0answers
824 views

Read error (Connection reset by peer) in headers on Vagrant

Trying to install aaPanel into a vagrant machine. vagrant box used is Scotch box 3.5 which preloaded with php v7.0. I've updated php to v7.4 using sury's ppa and working fine. I've used given ...
0
votes
0answers
13 views

Is there a way to automatically handle a source file type change from file to directory using wget?

I am using wget to mirror a website recursively (-r) with file changes only (-N), and have run into an issue for which I haven't been able to figure out a solution, and I was wondering if anyone out ...
0
votes
1answer
185 views

How to save output file to /dev/null with axel command?

I want to download a file from another server and then save it to /dev/null in order to increase monthly traffic of the first server. As man axel says, I should either use -o file or --output=file, ...
0
votes
0answers
21 views

Retransmissions, packet loss and impact in throughput

I'm working with a virtualized network where a client VM will download a website from a web server VM, through a router VM, using wget. So, client <---> router <---> web server. I'm using ...
0
votes
0answers
66 views

How to download all files from an FTP directory?

I am not experienced in unix or downloading files from FTP, html etc. so haven't been sure if I can safely use some other examples/questions given here. I am looking to download files from an ftp ...
0
votes
0answers
276 views

curl + wget cannot make https connections

My Ubuntu 16.04 system has the following problem: wget and curl hang indefinitely when connecting to a server with HTTPS Both programs work fine with HTTP Example for curl: curl -vv https://google....
1
vote
1answer
509 views

curl and wget fail to verify wildcard Let's Encrypt certificate, but browsers work fine

We've obtained a wildcard certificate from Let's Encrypt for civility.social and *.civility.social, using certbot. This works fine on all browsers and with curl and wget when verifying https://...
0
votes
0answers
16 views

Resuming download from multiple sites

Looking for any tools which downloads the single file hosted in different multiple websites matching the same size, version, name which includes matching AOO, KEYS and PGP/ASC keys and having SHA256 / ...
0
votes
0answers
31 views

scripting download of teradata files

I'm trying to automate the build of a server and am having troubles scripting out the download of the teradata tools and utilities for linux from this site https://downloads.teradata.com/download/...
1
vote
1answer
246 views

How To Use Wget Command To Check Server Cache Settings

I'm having trouble with a server side cache with an OpenLiteSpeed server. I've tried the following .htaccess trick to try and turn this cache off. Can you tell me a wget command I can use to test my ...
0
votes
1answer
28 views

wget specific directory with downloading whole path

I try to download data using this command. wget -r --no-parent http://myserver/username/data/ This leaves me with three folders on my server, namely: myserver/username/data/ How do I download only ...
0
votes
2answers
215 views

Use wget on a cluster with ssh-tunnel [closed]

Normally I can sercure copy files from one machine to another using > scp -oProxyJump=user@login.node.org ssh user@main.node.org:/home/user/my_files/* . which is very slow for large data sets. I ...
0
votes
2answers
45 views

Why ping statment doesn't work on this Linux Ubuntu VM? [closed]

I am not so into networking and I have the following doubt. They provide me a Linux Ubuntu 18.04.3 LTS VM on which I have to install some tools. The first thing that I tried was related to the ...
0
votes
1answer
89 views

Connect to ubuntu port from docker container

I have a application running on port 8180 on my local machine When I try to wget http://192.168.1.12:8180/auth from docker container, I get following error --2019-10-10 08:17:03-- http://192.168.1....
0
votes
1answer
54 views

How can I force `wget` to clobber a previously-downloaded file?

I'm trying to force wget to clobber a download and I haven't been able to make it do it with what I can find in the man pages. I'd like something cleaner than writing to and redirecting STDOUT. Anyone ...
1
vote
1answer
652 views

PASV FTP fails for wget/ftp but works for curl/lftp

I have a server running Centos 7 and vsftpd-3.0.3 (installed from source to get the latest version, though everything I describe was also applicable to the vsftpd-3.0.2 version that was previously ...
6
votes
1answer
499 views

Why does Amazon Linux 2 have older version of wget vs. Amazon Linux 1?

We noticed that wget on Amazon Linux 2 is v1.14, but on the older version of Amazon Linux, it was v1.18. Any idea why this would be? I'm fairly certain we installed from yum in both cases. Is there ...
0
votes
1answer
417 views

Git clone, wget, ping not working on GCP server all of a sudden

My Google Cloud Platform server is suddenly unable to download anything via wget, git clone, and unable to apt-get install anything. --2019-05-07 14:48:02-- http://google.com/ Resolving google.com (...
-3
votes
1answer
2k views

Can I assign the special IP address when I use wget?

I have a multi-IP addresses Server, a main IP 102.1.1.1 and a C segment 103.1.1.0/24 IP addresses on it. when I use wget www.google.com can I assign my special ip address in my Server like ping? ...
0
votes
0answers
3k views

CentOS 7: curl and wget not working

My problem is that curl and wget usually fail to retrieve more than just the first part of any given file, and often nothing at all. Both have worked previously on the same server, and nothing else on ...
1
vote
1answer
1k views

Upgrading wget on Ubuntu 16.04 server

I'm trying to upgrade my copy of wget on the server. Currently its running: 1.17.1-1ubuntu1.4 The problem is that I need 1.19 so I can get a new feature: http://tomszilagyi.github.io/2017/02/Wget-...
19
votes
6answers
9k views

Why the discrepancy between Speedtest and Wget?

My customer complains about low internet speeds. When measured with Speedtest.net speeds are acceptable. Periodic measured downloads are 10% to 30% of the nominal speed. I cannot explain that. Some ...
4
votes
2answers
3k views

Why does curl work with a specific https site, but wget has problems with certificates?

Centos 6. After updating recently ca-certificates I run into some troubles. I have the most recent versions of curl for centos 6: -bash-4.1$ curl -V -v curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7....
0
votes
0answers
84 views

create Linux user to give only wget permission

I own a dedicated server and I'm working on my server along some colleagues. I want to create some users for my colleagues which only be able to use wget command. I mean I don't want to give them ...
2
votes
2answers
346 views

`wget`-ting a website for “local” browsing on a different domain

I need to mirror a website and deploy the copy under a different domain name. The mirroring procedure should be all automatic, so that I can update the copy on a regular basis with cron. The mirror ...
0
votes
2answers
1k views

What does the -c in wget -c Linux command mean?

I am installing Artifactory on an Ubuntu server, and trying to add JFrog repository key. The command is: wget -c -O- "https://bintray.com/user/downloadSubjectPublicKey?username=jfrog" | sudo apt-key ...
2
votes
1answer
532 views

Automated export of phpMyAdmin SQL download with wget

I'm using the following script to attempt to download an SQL export from a phpMyAdmin installation (adapted from this Stackoverflow question) read -p "Username: " USERNAME read -p "Password: " ...
2
votes
0answers
9k views

How to Enable JavaScript for wget in linux for grabbing a website?

I use wget like this for save site : wget --page-requisites --no-parent --mirror http://example.com/index.html -P /home/ in Some cases It does NOT work, Error is : This site requires Javascript to ...
3
votes
3answers
15k views

Fixing wget certificates

Basically, I was testing different versions of OpenSSL, and somewhere in the process broke wget's path to certificates. wget https://www.google.com Results now in: ERROR: cannot verify www.google....
0
votes
1answer
2k views

wget download only certain folders from site

I want to download this site with wget, and to do this I'm using this command: wget.exe --no-clobber --no-parent -r --convert-links --page-requisites -U Mozilla "http://www.s-manuals.com/smd/" That'...
0
votes
1answer
4k views

Why can't I get wget to pipe standard output to /dev/null?

When I run this command it works as expected: wget -p https://www.google.com/ After running this command I check the error code with echo $? This returns 0, indicating that last command ran ...
-1
votes
1answer
88 views

stdout and stderr from bash execution to other machine?

Is it possible to send stderr or stdout from a linux machine to ubuntu linux machine using at best wget or other common package at worse ( hopefully not ssh access ) ? So far i can find a way to "...
2
votes
1answer
1k views

wget hangs/freezes when downloading file into NFS

I'm running some experiments with Amazon EFS (General purpose) and EC2 and I have the issue that EFS seems to act unstable. Commands that involve the mounted file system would hang or freeze. For ...
0
votes
1answer
481 views

Using wget to download GlobColour data from hermes ftp server

I want to download the netCDF products from ftp://ftp.hermes.acri.fr. I used wget in cygwin package for Windows systems. My link address for accessing to the data is ftp://ftp.hermes.acri.fr/691969751 ...
0
votes
1answer
542 views

FTP access error in EC2

I have 2 servers with windows and ubuntu running in Amazon EC2. I created the FTP server using plesk in windows and used vsftpd in the ubuntu. I have enabled all TCP ports in both the servers. The ...
4
votes
1answer
4k views

How to download a file and preserver original permissions using wget

How can I download a file that has already executable permissions 755 in another server. I want to: wget https://example.com/pub/register.sh --no-check-certificate and preserver permission register....
3
votes
3answers
5k views

nginx block curl and wget

I have nginx webserver. I have a rich content site and i found that some malicious bots are trying to crawl my content. I blocked any curl or wget coming to my site like this if ($http_user_agent ~* ...
0
votes
0answers
355 views

(mac) Need to automatically backup entire FTP server and create an archive

Is there a way to grab all files from an FTP server using wget (or similar), then create a zip or tar of the downloaded files? I would like to do this with one command or using Automator, and to ...
4
votes
1answer
7k views

Cannot get cURL or wget to validate some SSL certificates

I've noticed that our link checker, which uses cURL, fails more and more often to validate SSL certificates. I'm trying to get to the bottom of this. https://www.bgetem.de/, for instance, opens just ...
-1
votes
1answer
53 views

Stop wget clobbering local file is server unavailable

I have a cron job on a server which once a day uses wget to download "Earth Orientation Parameters" and leapsecond data from a NASA server. Specifically: wget https://gemini.gsfc.nasa.gov/500/oper/...
0
votes
0answers
931 views

Download WordPress site via wget

We have a unique situation whereby we are going to be hosting a website within a company's private network. They have given me RDC access. However, the remote machine has no internet access, so I am ...
2
votes
1answer
810 views

How to set index.html to redirect wget query to an other page/file?

If my server is at example.com, running $ wget example.com will only download the file index.html. How can I make wget either download another file instead of index.html download this file along ...

1
2 3 4 5 6