Questions tagged [wget]

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

Filter by
Sorted by
Tagged with
0 votes
2 answers
73 views

use wget in bash script - how to get downloaded file name? (using content-disposition)

i am using wget to download some file in a bash script. i need to use it like this: wget -t 20 --content-disposition $link the problem here is, that i cannot specify the output file since it comes ...
38 votes
7 answers
90k views

Wget a script and run it

I want to download a script from: http://dl.dropbox.com/u/11210438/flockonus-stack.sh and execute it. My guess is, to use wget renaming it, chmod it, and execute. What are the commands for doing ...
0 votes
0 answers
57 views

YARN resource manager "HTTP request sent, awaiting response..."

we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine in our cluster YARN ...
0 votes
1 answer
201 views

Prometheus Blackbox http_2xx returning 403 since webserver migrated to StackCP

I had working Prometheus Blackbox Exporter http_2xx checks monitoring various web servers. Then the web hosting provider migrated from cPanel to Stack CP. Since then all the http_2xx just return 403 (...
0 votes
0 answers
26 views

download file from heptapod - curl or wget

With my install script for Tryton ERP, I need to download files from heptapod.net. As on my target servers git normally is not needed, I should use standard tools like wget or curl. Could somebody ...
2 votes
2 answers
1k views

wget 5xx error code handling

I'm executing a shell script that utilizes wget to retrieve data from the web. I'm wondering if there's a way to instruct wget to terminate the process immediately upon encountering a 500 error code.
0 votes
2 answers
1k views

Download with Wget only if new version

Good morning, I have a custom software that updates with a custom script.sh. Part of the file goes something like this: if [[ $software == A ]] then echo "downloading package..." ...
0 votes
1 answer
227 views

wget/curl from internal network (web server) to external ip address (web server) connection failed

for some monitoring purposes on my own software i want to call wget with the "external" UR of the webserver, so my call will look like wget https://www.mydomain.de/path But this call fails ...
25 votes
3 answers
40k views

Wget: Is there a way to pass username and password from a file?

Is there a way to pass the username and password from a file instead of the command line via --user and --password? Background: I want to run wget via cron and don't want the username/password show ...
0 votes
0 answers
46 views

wget/FTP downloading issue

I am using the command wget -r ftp://xyz/../ to download a directory on my wsl. It is a huge directory, so it takes hours to finish. My internet got disabled for a little bit, and the process ...
1 vote
2 answers
894 views

How to save output file to /dev/null with axel command?

I want to download a file from another server and then save it to /dev/null in order to increase monthly traffic of the first server. As man axel says, I should either use -o file or --output=file, ...
13 votes
7 answers
16k views

wget recursive download, but I don't want to follow all links

I'm trying to mirror a website using wget, but I don't want to download lots of files, so I'm using wget's --reject option to not save all the files. However wget will still download all the files and ...
0 votes
0 answers
247 views

wget doesn't accept HTTPS certificate (from letsencrypt) on some computers but does on others

I have the following situation: On my www server (Apache2, Debian armhf) I have HTTPS certificate from letsencrypt.org by certbot. When accessing the server from a browser (Firefox, Chromium) ...
0 votes
2 answers
251 views

apache remove wget for better security

Im no expert at IT or apache but I have used linux for years. Ive done my best to harden/secure my server online which I use for a mobile app which needs to serve up its content etc. Ive noticed some ...
276 votes
6 answers
566k views

how to download the ssl certificate from a website?

I want to download the ssl certificate from, say https://www.google.com, using wget or any other commands. Any unix command line? wget or openssl?
1 vote
0 answers
35 views

YARN + how to debug wget

we are testing with wget VIA port 8088 the connection from ResourceManager02 to ResourceManager01 both Resource Managers are part of YARN service , and each resource manager service installed on RHEL ...
1 vote
0 answers
55 views

assign result and status code from wget to variables

Can I get the response and status code and assign them to variables? # wget https://www.google.com --quiet --server-response 1> >(body=$(</dev/stdin)) 2> >(grep HTTP | awk '{print $2}' |...
-3 votes
2 answers
5k views

Can I assign the special IP address when I use wget?

I have a multi-IP addresses Server, a main IP 102.1.1.1 and a C segment 103.1.1.0/24 IP addresses on it. when I use wget www.google.com can I assign my special ip address in my Server like ping? ...
0 votes
1 answer
1k views

wget cannot find trusted certificate

On my RedHat 8 system, wget fails with error below: wget -v https://10.81.72.97/repo/packages/installer-5.1-1.x86_64.rpm --2022-12-05 19:40:00-- https://10.81.72.97/repo/packages/installer-5.1-1....
0 votes
2 answers
150 views

Cronjob to visit each url of a list but not save them

I have a txt file list.txt that have bunch of urls, one every line. I want to set a cron job that wget/curl each url of the file once a day but does not save them in the computer. I tried to run this ...
0 votes
3 answers
2k views

IPTables: apt-get and wget not working

When resetting IPTables, the apt-get and wget command functions correctly and also downloads what I want. But once I activate this firewall, it isn't functional. Pings still work. I want to allow all ...
2 votes
2 answers
3k views

Cron getting timeout when attempting to access url (wget).

I'm sure this is simple and I've been digging but no answer is quite as specific as I need it to be. The goal is easy, have Cron hit a url on my server every 5 minutes. All of that is set up and ...
19 votes
5 answers
16k views

Why is only one wget command working in my crontab?

I wish to fetch content from a PHP script on my server two times a day, altering a query variable lang to set what language we want, and save this content in two language specific files. This is my ...
1 vote
2 answers
2k views

Trying to mirror my website with wget, but nofollow attribute is found , and I cannot download anything more than index.html

I am running a wordpress site on a Ubuntu 20.04 based LEMP server. I have the pagespeed plugin enabled, and in order to force it to cache my website, I am using wget from a different box to mirror the ...
0 votes
1 answer
690 views

Move files from a remote ftp server to a local

I need to pull files from an ftp server regularly and have found that I can get the files easily enough using wget wget -m --user=yyy --password=xxxx ftps://host.com.au and that works really nicely. ...
2 votes
3 answers
6k views

wget and port forwarding

I have a weird problem. I can run this command just fine from various servers: wget --debug '--http-user=USER123' '--http-passwd=PASSWORD' http://GW-BOX:9091/weijhkdsvn/v9_odbc//CRONTAB.2014020 ...
4 votes
5 answers
31k views

Fixing wget certificates

Basically, I was testing different versions of OpenSSL, and somewhere in the process broke wget's path to certificates. wget https://www.google.com Results now in: ERROR: cannot verify www.google....
11 votes
4 answers
12k views

How to refresh an online website mirror created with `wget --mirror`?

One month ago, I used "wget --mirror" to create a mirror of our public website for temporary use during an upcoming scheduled maintenance window. Our primary website runs HTML, PHP & MySQL, but ...
1 vote
1 answer
3k views

curl and wget fail to verify wildcard Let's Encrypt certificate, but browsers work fine

We've obtained a wildcard certificate from Let's Encrypt for civility.social and *.civility.social, using certbot. This works fine on all browsers and with curl and wget when verifying https://...
0 votes
1 answer
4k views

wget is not able to get the correct ssl certifiicate expiration date

When I execute this wget -O /root/installer.php https://www.grscripts.com/install/installer.deluxe I receive this error ERROR: cannot verify https://www.grscripts.com's/ certificate, issued by ‘/C=...
1 vote
2 answers
6k views

vshpere wget: bad address 'releases.ubuntu.com'

on a raw vsphere esxi 6 without any vm installed i have to setup vm's and ... remotely so i decided to open ssh to esxi and download an Ubuntu iso using wget this way: wget http://releases.ubuntu....
0 votes
0 answers
73 views

wget: Edit broken links while crawling before visiting them

Situation: I want to mirror an old website. This website is on https://example.com/website/. The website uses absolute links to http://www.example.com/website/. Problem: For whatever reason, wget ...
0 votes
2 answers
7k views

Downloading a file using Windows CMD line with curl/wget

I have a client [Windows 10 VM] and a server [say a linux based VM]. I have Apache running on the Linux Server. I have a file on the linux server that I want to download on my windows client. I want ...
0 votes
1 answer
148 views

ICS Calendar File Does not Exist

I'm trying to load this calendar: https://bushnellbeacons.com/calendar.ics to another internal websystem. The calendar feed is fine - I can click the link above and download the feed. The problem is ...
0 votes
1 answer
153 views

wget does not recurse when piping the output to stdout [closed]

I want to download webpages recursively and pipe the output to a filter. I am using: wget -qm -O- http://mywebsite.com/initialpath.php | ./filter But wget stops downloading after the first page and ...
0 votes
1 answer
411 views

How can I tell if wget can support LetsEncrypt's new ISRG Root X1 certificate after DST Root CA X3 Expiration

A few weeks ago (Sept 2021), LetsEncrypt changed how their certificates were signed, which can affect some older programmes and clients. I have a standard Apache webserver (stock v2.4.41 from ubuntu ...
5 votes
2 answers
14k views

How can I get a static download link for a Microsoft MSDN Subscriber Download?

I need to be able to download MSDN ISO files directly from PowerShell. How can I do this? I can view the downloads site at: https://msdn.microsoft.com/subscriptions/securedownloads/ however, these ...
0 votes
1 answer
833 views

PXE boot pmagic with wget or TFTP after kernel

I wish to boot parted magic by PXE. I have read this article on the topic: https://partedmagic.com/pxe/ where it states that you can either load the entire iso, prepare the squashfs file and load it ...
0 votes
1 answer
3k views

wget failed: Connection timed out

I have the following command to copy the website, as it tried to hit sun.com it got connection timed out. I would like the wget to exclude the sun.com so that wget would proceed to the next thing. ...
0 votes
1 answer
3k views

Troubleshooting nginx requests not reaching server

When I browse from other servers using wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" --spider http://server_ip/page --no-hsts This is the response Spider mode enabled. ...
7 votes
5 answers
12k views

"Issued certificate not yet valid." with wget?

I am getting the following error when using wget to fetch a file: Issued certificate not yet valid. To connect to www.atlassian.com insecurely, use ‘--no-check-certificate’. Testing with openssl ...
-1 votes
2 answers
101 views

Backup my webspace to NAS [closed]

First of all: I don't know if this is the correct place to ask, please tell me where I can go with this before you remove my post, that would be great, thank you! ​ What I want to do (and what my ...
0 votes
1 answer
2k views

Can I make curl (or wget) ignore a specific TLS error? (bypassing expired certificate without disabling cert validation)

I'm working on a script which depends on a remote API endpoint which I do not control. Today, my script stopped working, because the endpoint's SSL certificate expired today and they haven't yet fixed ...
8 votes
3 answers
34k views

Alternatives to CURL, WGET

Is anyone aware of any alternatives to curl and wget? The key functional requirement I'm looking for is to be able to execute an HTTPS GET request against a known URL. The reason I'm looking for an ...
18 votes
3 answers
41k views

How to exclude certain directories while using wget?

I'd like to download a directory from a FTP, which contains some source codes. Initially, I did this: wget -r ftp://path/to/src Unfortunately, the directory itself is a result of a SVN checkout, so ...
0 votes
1 answer
1k views

wget -m delete local files that are not existing anymore on the server

I use wget -m to mirror my webspace to my nas. So far it's working great, but I noticed that when a file on the server got deleted, it will be still there locally. Is there any way to delete the local ...
8 votes
1 answer
32k views

How do I fix certificate errors when running wget on an HTTPS URL in Cygwin Windows?

I have Cygwin installed in my windows system. I am trying to execute wget command for a website but I am getting the following error message: ERROR: The certificate of `example.com' is not trusted. ...
29 votes
4 answers
153k views

Why wget doesn't verify SSL certificates?

I have a problem with my Fedora 8 installation. It looks that wget doesn't know how to verify SSL certificates any more. It's strange because I have another Fedora 8 box which I believe has the same ...
0 votes
0 answers
100 views

Wget- Skip hosts that are down

I'm using wget to bulk download a website, and it grabs files from other servers, however, some hosts are down. All wget does is just hang there, so I'd like it to just skip these websites. Is there ...
2 votes
0 answers
159 views

Is there a tool I can use to test out multiple vpn tunnels in one machine?

I use wget -q -O https://checkmyip.com/csv to determine if I am connected to a VPN but I want to verify multiple VPN tunnels on the same machine. Is there a tool that can allow me to send a command ...

1
2 3 4 5 6