Questions tagged [wget]

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

Filter by
Sorted by
Tagged with
0 votes
2 answers
55 views

use wget in bash script - how to get downloaded file name? (using content-disposition)

i am using wget to download some file in a bash script. i need to use it like this: wget -t 20 --content-disposition $link the problem here is, that i cannot specify the output file since it comes ...
Hannes F's user avatar
  • 101
0 votes
0 answers
55 views

YARN resource manager "HTTP request sent, awaiting response..."

we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine in our cluster YARN ...
King David's user avatar
0 votes
1 answer
175 views

Prometheus Blackbox http_2xx returning 403 since webserver migrated to StackCP

I had working Prometheus Blackbox Exporter http_2xx checks monitoring various web servers. Then the web hosting provider migrated from cPanel to Stack CP. Since then all the http_2xx just return 403 (...
codlord's user avatar
  • 125
0 votes
0 answers
26 views

download file from heptapod - curl or wget

With my install script for Tryton ERP, I need to download files from heptapod.net. As on my target servers git normally is not needed, I should use standard tools like wget or curl. Could somebody ...
herrdeh's user avatar
0 votes
1 answer
223 views

wget/curl from internal network (web server) to external ip address (web server) connection failed

for some monitoring purposes on my own software i want to call wget with the "external" UR of the webserver, so my call will look like wget https://www.mydomain.de/path But this call fails ...
Opa114's user avatar
  • 113
0 votes
0 answers
45 views

wget/FTP downloading issue

I am using the command wget -r ftp://xyz/../ to download a directory on my wsl. It is a huge directory, so it takes hours to finish. My internet got disabled for a little bit, and the process ...
Archit Gupta's user avatar
0 votes
0 answers
241 views

wget doesn't accept HTTPS certificate (from letsencrypt) on some computers but does on others

I have the following situation: On my www server (Apache2, Debian armhf) I have HTTPS certificate from letsencrypt.org by certbot. When accessing the server from a browser (Firefox, Chromium) ...
bicyclesonthemoon's user avatar
0 votes
2 answers
248 views

apache remove wget for better security

Im no expert at IT or apache but I have used linux for years. Ive done my best to harden/secure my server online which I use for a mobile app which needs to serve up its content etc. Ive noticed some ...
Tim's user avatar
  • 203
1 vote
0 answers
35 views

YARN + how to debug wget

we are testing with wget VIA port 8088 the connection from ResourceManager02 to ResourceManager01 both Resource Managers are part of YARN service , and each resource manager service installed on RHEL ...
King David's user avatar
1 vote
0 answers
53 views

assign result and status code from wget to variables

Can I get the response and status code and assign them to variables? # wget https://www.google.com --quiet --server-response 1> >(body=$(</dev/stdin)) 2> >(grep HTTP | awk '{print $2}' |...
Richard Barraclough's user avatar
0 votes
1 answer
1k views

wget cannot find trusted certificate

On my RedHat 8 system, wget fails with error below: wget -v https://10.81.72.97/repo/packages/installer-5.1-1.x86_64.rpm --2022-12-05 19:40:00-- https://10.81.72.97/repo/packages/installer-5.1-1....
Amol's user avatar
  • 103
0 votes
2 answers
147 views

Cronjob to visit each url of a list but not save them

I have a txt file list.txt that have bunch of urls, one every line. I want to set a cron job that wget/curl each url of the file once a day but does not save them in the computer. I tried to run this ...
Sahriar Saikat's user avatar
1 vote
2 answers
2k views

Trying to mirror my website with wget, but nofollow attribute is found , and I cannot download anything more than index.html

I am running a wordpress site on a Ubuntu 20.04 based LEMP server. I have the pagespeed plugin enabled, and in order to force it to cache my website, I am using wget from a different box to mirror the ...
DanRan's user avatar
  • 123
0 votes
1 answer
679 views

Move files from a remote ftp server to a local

I need to pull files from an ftp server regularly and have found that I can get the files easily enough using wget wget -m --user=yyy --password=xxxx ftps://host.com.au and that works really nicely. ...
Peter Nunn's user avatar
0 votes
1 answer
4k views

wget is not able to get the correct ssl certifiicate expiration date

When I execute this wget -O /root/installer.php https://www.grscripts.com/install/installer.deluxe I receive this error ERROR: cannot verify https://www.grscripts.com's/ certificate, issued by ‘/C=...
gr68's user avatar
  • 123
0 votes
0 answers
73 views

wget: Edit broken links while crawling before visiting them

Situation: I want to mirror an old website. This website is on https://example.com/website/. The website uses absolute links to http://www.example.com/website/. Problem: For whatever reason, wget ...
user136036's user avatar
0 votes
2 answers
6k views

Downloading a file using Windows CMD line with curl/wget

I have a client [Windows 10 VM] and a server [say a linux based VM]. I have Apache running on the Linux Server. I have a file on the linux server that I want to download on my windows client. I want ...
Software Fan's user avatar
0 votes
1 answer
148 views

ICS Calendar File Does not Exist

I'm trying to load this calendar: https://bushnellbeacons.com/calendar.ics to another internal websystem. The calendar feed is fine - I can click the link above and download the feed. The problem is ...
SteadH's user avatar
  • 676
0 votes
1 answer
151 views

wget does not recurse when piping the output to stdout [closed]

I want to download webpages recursively and pipe the output to a filter. I am using: wget -qm -O- http://mywebsite.com/initialpath.php | ./filter But wget stops downloading after the first page and ...
chqrlie's user avatar
  • 111
0 votes
1 answer
405 views

How can I tell if wget can support LetsEncrypt's new ISRG Root X1 certificate after DST Root CA X3 Expiration

A few weeks ago (Sept 2021), LetsEncrypt changed how their certificates were signed, which can affect some older programmes and clients. I have a standard Apache webserver (stock v2.4.41 from ubuntu ...
Amandasaurus's user avatar
  • 32.2k
0 votes
1 answer
830 views

PXE boot pmagic with wget or TFTP after kernel

I wish to boot parted magic by PXE. I have read this article on the topic: https://partedmagic.com/pxe/ where it states that you can either load the entire iso, prepare the squashfs file and load it ...
FalcoGer's user avatar
  • 136
0 votes
1 answer
3k views

wget failed: Connection timed out

I have the following command to copy the website, as it tried to hit sun.com it got connection timed out. I would like the wget to exclude the sun.com so that wget would proceed to the next thing. ...
bal's user avatar
  • 1
0 votes
1 answer
3k views

Troubleshooting nginx requests not reaching server

When I browse from other servers using wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" --spider http://server_ip/page --no-hsts This is the response Spider mode enabled. ...
Ajay Singh's user avatar
-1 votes
2 answers
101 views

Backup my webspace to NAS [closed]

First of all: I don't know if this is the correct place to ask, please tell me where I can go with this before you remove my post, that would be great, thank you! ​ What I want to do (and what my ...
jona's user avatar
  • 113
0 votes
2 answers
1k views

Download with Wget only if new version

Good morning, I have a custom software that updates with a custom script.sh. Part of the file goes something like this: if [[ $software == A ]] then echo "downloading package..." ...
Perovic 's user avatar
0 votes
1 answer
1k views

wget -m delete local files that are not existing anymore on the server

I use wget -m to mirror my webspace to my nas. So far it's working great, but I noticed that when a file on the server got deleted, it will be still there locally. Is there any way to delete the local ...
jona's user avatar
  • 113
0 votes
1 answer
2k views

Can I make curl (or wget) ignore a specific TLS error? (bypassing expired certificate without disabling cert validation)

I'm working on a script which depends on a remote API endpoint which I do not control. Today, my script stopped working, because the endpoint's SSL certificate expired today and they haven't yet fixed ...
Wug's user avatar
  • 151
0 votes
0 answers
99 views

Wget- Skip hosts that are down

I'm using wget to bulk download a website, and it grabs files from other servers, however, some hosts are down. All wget does is just hang there, so I'd like it to just skip these websites. Is there ...
Honest's user avatar
  • 1
2 votes
0 answers
155 views

Is there a tool I can use to test out multiple vpn tunnels in one machine?

I use wget -q -O https://checkmyip.com/csv to determine if I am connected to a VPN but I want to verify multiple VPN tunnels on the same machine. Is there a tool that can allow me to send a command ...
LeanMan's user avatar
  • 181
2 votes
2 answers
395 views

Can't serve static files with ampersand in name

I have a website mirrored with wget and I'd like to serve it via Apache. The problem is that I can't access files with special chars. For example, I have files with names like this: $ ls -N components/...
Grzegorz Adam Kowalski's user avatar
0 votes
1 answer
345 views

wget website recusively from localhost without using bandwith

i'm looking to download recusively my wordpress website into static using wget, the problem is that whenever i do that, it's using too much bandwith (3.5gb) even though i end up downloading 20mb which ...
logax's user avatar
  • 139
0 votes
0 answers
3k views

Read error (Connection reset by peer) in headers on Vagrant

Trying to install aaPanel into a vagrant machine. vagrant box used is Scotch box 3.5 which preloaded with php v7.0. I've updated php to v7.4 using sury's ppa and working fine. I've used given ...
Vishwa's user avatar
  • 101
1 vote
2 answers
885 views

How to save output file to /dev/null with axel command?

I want to download a file from another server and then save it to /dev/null in order to increase monthly traffic of the first server. As man axel says, I should either use -o file or --output=file, ...
Saeed's user avatar
  • 162
0 votes
0 answers
1k views

How to download all files from an FTP directory?

I am not experienced in unix or downloading files from FTP, html etc. so haven't been sure if I can safely use some other examples/questions given here. I am looking to download files from an ftp ...
DN1's user avatar
  • 101
0 votes
0 answers
1k views

curl + wget cannot make https connections

My Ubuntu 16.04 system has the following problem: wget and curl hang indefinitely when connecting to a server with HTTPS Both programs work fine with HTTP Example for curl: curl -vv https://google....
oarfish's user avatar
  • 51
1 vote
1 answer
3k views

curl and wget fail to verify wildcard Let's Encrypt certificate, but browsers work fine

We've obtained a wildcard certificate from Let's Encrypt for civility.social and *.civility.social, using certbot. This works fine on all browsers and with curl and wget when verifying https://...
Civility Social's user avatar
0 votes
0 answers
54 views

scripting download of teradata files

I'm trying to automate the build of a server and am having troubles scripting out the download of the teradata tools and utilities for linux from this site https://downloads.teradata.com/download/...
Stromweld's user avatar
1 vote
1 answer
1k views

How To Use Wget Command To Check Server Cache Settings

I'm having trouble with a server side cache with an OpenLiteSpeed server. I've tried the following .htaccess trick to try and turn this cache off. Can you tell me a wget command I can use to test my ...
ServerChecker's user avatar
0 votes
1 answer
368 views

wget specific directory with downloading whole path

I try to download data using this command. wget -r --no-parent http://myserver/username/data/ This leaves me with three folders on my server, namely: myserver/username/data/ How do I download only ...
user2300940's user avatar
0 votes
2 answers
587 views

Use wget on a cluster with ssh-tunnel [closed]

Normally I can sercure copy files from one machine to another using > scp [email protected] ssh [email protected]:/home/user/my_files/* . which is very slow for large data sets. I ...
user3166083's user avatar
0 votes
2 answers
789 views

Why ping statment doesn't work on this Linux Ubuntu VM? [closed]

I am not so into networking and I have the following doubt. They provide me a Linux Ubuntu 18.04.3 LTS VM on which I have to install some tools. The first thing that I tried was related to the ...
AndreaNobili's user avatar
0 votes
1 answer
1k views

Connect to ubuntu port from docker container

I have a application running on port 8180 on my local machine When I try to wget http://192.168.1.12:8180/auth from docker container, I get following error --2019-10-10 08:17:03-- http://192.168.1....
Sazzad Islam's user avatar
0 votes
1 answer
599 views

How can I force `wget` to clobber a previously-downloaded file?

I'm trying to force wget to clobber a download and I haven't been able to make it do it with what I can find in the man pages. I'd like something cleaner than writing to and redirecting STDOUT. Anyone ...
sjlxndr's user avatar
  • 36
1 vote
1 answer
2k views

PASV FTP fails for wget/ftp but works for curl/lftp

I have a server running Centos 7 and vsftpd-3.0.3 (installed from source to get the latest version, though everything I describe was also applicable to the vsftpd-3.0.2 version that was previously ...
dan's user avatar
  • 333
8 votes
1 answer
1k views

Why does Amazon Linux 2 have older version of wget vs. Amazon Linux 1?

We noticed that wget on Amazon Linux 2 is v1.14, but on the older version of Amazon Linux, it was v1.18. Any idea why this would be? I'm fairly certain we installed from yum in both cases. Is there ...
boozedog's user avatar
0 votes
1 answer
1k views

Git clone, wget, ping not working on GCP server all of a sudden

My Google Cloud Platform server is suddenly unable to download anything via wget, git clone, and unable to apt-get install anything. --2019-05-07 14:48:02-- http://google.com/ Resolving google.com (...
Dominique's user avatar
  • 101
-3 votes
2 answers
5k views

Can I assign the special IP address when I use wget?

I have a multi-IP addresses Server, a main IP 102.1.1.1 and a C segment 103.1.1.0/24 IP addresses on it. when I use wget www.google.com can I assign my special ip address in my Server like ping? ...
aircraft's user avatar
  • 396
0 votes
0 answers
6k views

CentOS 7: curl and wget not working

My problem is that curl and wget usually fail to retrieve more than just the first part of any given file, and often nothing at all. Both have worked previously on the same server, and nothing else on ...
SuperDog's user avatar
1 vote
1 answer
6k views

Upgrading wget on Ubuntu 16.04 server

I'm trying to upgrade my copy of wget on the server. Currently its running: 1.17.1-1ubuntu1.4 The problem is that I need 1.19 so I can get a new feature: http://tomszilagyi.github.io/2017/02/Wget-...
Andrew Newby's user avatar
  • 1,154
18 votes
6 answers
15k views

Why the discrepancy between Speedtest and Wget?

My customer complains about low internet speeds. When measured with Speedtest.net speeds are acceptable. Periodic measured downloads are 10% to 30% of the nominal speed. I cannot explain that. Some ...
Johannes Linkels's user avatar

1
2 3 4 5 6