Questions tagged [wget]
GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.
294
questions
0
votes
2
answers
55
views
use wget in bash script - how to get downloaded file name? (using content-disposition)
i am using wget to download some file in a bash script. i need to use it like this:
wget -t 20 --content-disposition $link
the problem here is, that i cannot specify the output file since it comes ...
0
votes
0
answers
55
views
YARN resource manager "HTTP request sent, awaiting response..."
we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine
in our cluster YARN ...
0
votes
1
answer
175
views
Prometheus Blackbox http_2xx returning 403 since webserver migrated to StackCP
I had working Prometheus Blackbox Exporter http_2xx checks monitoring various web servers. Then the web hosting provider migrated from cPanel to Stack CP.
Since then all the http_2xx just return 403 (...
0
votes
0
answers
26
views
download file from heptapod - curl or wget
With my install script for Tryton ERP, I need to download files from heptapod.net. As on my target servers git normally is not needed, I should use standard tools like wget or curl.
Could somebody ...
0
votes
1
answer
223
views
wget/curl from internal network (web server) to external ip address (web server) connection failed
for some monitoring purposes on my own software i want to call wget with the "external" UR of the webserver, so my call will look like wget https://www.mydomain.de/path
But this call fails ...
0
votes
0
answers
45
views
wget/FTP downloading issue
I am using the command
wget -r ftp://xyz/../
to download a directory on my wsl. It is a huge directory, so it takes hours to finish. My internet got disabled for a little bit, and the process ...
0
votes
0
answers
241
views
wget doesn't accept HTTPS certificate (from letsencrypt) on some computers but does on others
I have the following situation:
On my www server (Apache2, Debian armhf) I have HTTPS certificate from letsencrypt.org by certbot.
When accessing the server from a browser (Firefox, Chromium) ...
0
votes
2
answers
248
views
apache remove wget for better security
Im no expert at IT or apache but I have used linux for years.
Ive done my best to harden/secure my server online which I use for a mobile app which
needs to serve up its content etc.
Ive noticed some ...
1
vote
0
answers
35
views
YARN + how to debug wget
we are testing with wget VIA port 8088 the connection from ResourceManager02 to ResourceManager01
both Resource Managers are part of YARN service , and each resource manager service installed on RHEL ...
1
vote
0
answers
53
views
assign result and status code from wget to variables
Can I get the response and status code and assign them to variables?
# wget https://www.google.com --quiet --server-response 1> >(body=$(</dev/stdin)) 2> >(grep HTTP | awk '{print $2}' |...
0
votes
1
answer
1k
views
wget cannot find trusted certificate
On my RedHat 8 system, wget fails with error below:
wget -v https://10.81.72.97/repo/packages/installer-5.1-1.x86_64.rpm
--2022-12-05 19:40:00-- https://10.81.72.97/repo/packages/installer-5.1-1....
0
votes
2
answers
147
views
Cronjob to visit each url of a list but not save them
I have a txt file list.txt that have bunch of urls, one every line. I want to set a cron job that wget/curl each url of the file once a day but does not save them in the computer.
I tried to run this ...
1
vote
2
answers
2k
views
Trying to mirror my website with wget, but nofollow attribute is found , and I cannot download anything more than index.html
I am running a wordpress site on a Ubuntu 20.04 based LEMP server.
I have the pagespeed plugin enabled, and in order to force it to cache my website, I am using wget from a different box to mirror the ...
0
votes
1
answer
679
views
Move files from a remote ftp server to a local
I need to pull files from an ftp server regularly and have found that I can get the files easily enough using wget
wget -m --user=yyy --password=xxxx ftps://host.com.au
and that works really nicely. ...
0
votes
1
answer
4k
views
wget is not able to get the correct ssl certifiicate expiration date
When I execute this
wget -O /root/installer.php https://www.grscripts.com/install/installer.deluxe
I receive this error
ERROR: cannot verify https://www.grscripts.com's/ certificate, issued by ‘/C=...
0
votes
0
answers
73
views
wget: Edit broken links while crawling before visiting them
Situation:
I want to mirror an old website. This website is on https://example.com/website/. The website uses absolute links to http://www.example.com/website/.
Problem:
For whatever reason, wget ...
0
votes
2
answers
6k
views
Downloading a file using Windows CMD line with curl/wget
I have a client [Windows 10 VM] and a server [say a linux based VM].
I have Apache running on the Linux Server.
I have a file on the linux server that I want to download on my windows client.
I want ...
0
votes
1
answer
148
views
ICS Calendar File Does not Exist
I'm trying to load this calendar:
https://bushnellbeacons.com/calendar.ics
to another internal websystem. The calendar feed is fine - I can click the link above and download the feed. The problem is ...
0
votes
1
answer
151
views
wget does not recurse when piping the output to stdout [closed]
I want to download webpages recursively and pipe the output to a filter. I am using:
wget -qm -O- http://mywebsite.com/initialpath.php | ./filter
But wget stops downloading after the first page and ...
0
votes
1
answer
405
views
How can I tell if wget can support LetsEncrypt's new ISRG Root X1 certificate after DST Root CA X3 Expiration
A few weeks ago (Sept 2021), LetsEncrypt changed how their certificates were signed, which can affect some older programmes and clients. I have a standard Apache webserver (stock v2.4.41 from ubuntu ...
0
votes
1
answer
830
views
PXE boot pmagic with wget or TFTP after kernel
I wish to boot parted magic by PXE.
I have read this article on the topic: https://partedmagic.com/pxe/
where it states that you can either load the entire iso, prepare the squashfs file and load it ...
0
votes
1
answer
3k
views
wget failed: Connection timed out
I have the following command to copy the website,
as it tried to hit sun.com it got connection timed out.
I would like the wget to exclude the sun.com so that wget would proceed to the next thing.
...
0
votes
1
answer
3k
views
Troubleshooting nginx requests not reaching server
When I browse from other servers using
wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" --spider http://server_ip/page --no-hsts
This is the response
Spider mode enabled. ...
-1
votes
2
answers
101
views
Backup my webspace to NAS [closed]
First of all: I don't know if this is the correct place to ask, please tell me where I can go with this before you remove my post, that would be great, thank you!
What I want to do (and what my ...
0
votes
2
answers
1k
views
Download with Wget only if new version
Good morning,
I have a custom software that updates with a custom script.sh.
Part of the file goes something like this:
if [[ $software == A ]]
then
echo "downloading package..."
...
0
votes
1
answer
1k
views
wget -m delete local files that are not existing anymore on the server
I use wget -m to mirror my webspace to my nas.
So far it's working great, but I noticed that when a file on the server got deleted, it will be still there locally.
Is there any way to delete the local ...
0
votes
1
answer
2k
views
Can I make curl (or wget) ignore a specific TLS error? (bypassing expired certificate without disabling cert validation)
I'm working on a script which depends on a remote API endpoint which I do not control.
Today, my script stopped working, because the endpoint's SSL certificate expired today and they haven't yet fixed ...
0
votes
0
answers
99
views
Wget- Skip hosts that are down
I'm using wget to bulk download a website, and it grabs files from other servers, however, some hosts are down. All wget does is just hang there, so I'd like it to just skip these websites. Is there ...
2
votes
0
answers
155
views
Is there a tool I can use to test out multiple vpn tunnels in one machine?
I use wget -q -O https://checkmyip.com/csv to determine if I am connected to a VPN but I want to verify multiple VPN tunnels on the same machine. Is there a tool that can allow me to send a command ...
2
votes
2
answers
395
views
Can't serve static files with ampersand in name
I have a website mirrored with wget and I'd like to serve it via Apache. The problem is that I can't access files with special chars. For example, I have files with names like this:
$ ls -N components/...
0
votes
1
answer
345
views
wget website recusively from localhost without using bandwith
i'm looking to download recusively my wordpress website into static using wget, the problem is that whenever i do that, it's using too much bandwith (3.5gb) even though i end up downloading 20mb which ...
0
votes
0
answers
3k
views
Read error (Connection reset by peer) in headers on Vagrant
Trying to install aaPanel into a vagrant machine. vagrant box used is Scotch box 3.5 which preloaded with php v7.0. I've updated php to v7.4 using sury's ppa and working fine.
I've used given ...
1
vote
2
answers
885
views
How to save output file to /dev/null with axel command?
I want to download a file from another server and then save it to /dev/null in order to increase monthly traffic of the first server.
As man axel says, I should either use -o file or --output=file, ...
0
votes
0
answers
1k
views
How to download all files from an FTP directory?
I am not experienced in unix or downloading files from FTP, html etc. so haven't been sure if I can safely use some other examples/questions given here.
I am looking to download files from an ftp ...
0
votes
0
answers
1k
views
curl + wget cannot make https connections
My Ubuntu 16.04 system has the following problem:
wget and curl hang indefinitely when connecting to a server with HTTPS
Both programs work fine with HTTP
Example for curl:
curl -vv https://google....
1
vote
1
answer
3k
views
curl and wget fail to verify wildcard Let's Encrypt certificate, but browsers work fine
We've obtained a wildcard certificate from Let's Encrypt for civility.social and *.civility.social, using certbot. This works fine on all browsers and with curl and wget when verifying https://...
0
votes
0
answers
54
views
scripting download of teradata files
I'm trying to automate the build of a server and am having troubles scripting out the download of the teradata tools and utilities for linux from this site https://downloads.teradata.com/download/...
1
vote
1
answer
1k
views
How To Use Wget Command To Check Server Cache Settings
I'm having trouble with a server side cache with an OpenLiteSpeed server. I've tried the following .htaccess trick to try and turn this cache off. Can you tell me a wget command I can use to test my ...
0
votes
1
answer
368
views
wget specific directory with downloading whole path
I try to download data using this command.
wget -r --no-parent http://myserver/username/data/
This leaves me with three folders on my server, namely: myserver/username/data/
How do I download only ...
0
votes
2
answers
587
views
Use wget on a cluster with ssh-tunnel [closed]
Normally I can sercure copy files from one machine to another using
> scp [email protected] ssh [email protected]:/home/user/my_files/* .
which is very slow for large data sets.
I ...
0
votes
2
answers
789
views
Why ping statment doesn't work on this Linux Ubuntu VM? [closed]
I am not so into networking and I have the following doubt.
They provide me a Linux Ubuntu 18.04.3 LTS VM on which I have to install some tools.
The first thing that I tried was related to the ...
0
votes
1
answer
1k
views
Connect to ubuntu port from docker container
I have a application running on port 8180 on my local machine
When I try to wget http://192.168.1.12:8180/auth from docker container,
I get following error
--2019-10-10 08:17:03-- http://192.168.1....
0
votes
1
answer
599
views
How can I force `wget` to clobber a previously-downloaded file?
I'm trying to force wget to clobber a download and I haven't been able to make it do it with what I can find in the man pages. I'd like something cleaner than writing to and redirecting STDOUT. Anyone ...
1
vote
1
answer
2k
views
PASV FTP fails for wget/ftp but works for curl/lftp
I have a server running Centos 7 and vsftpd-3.0.3 (installed from source to get the latest version, though everything I describe was also applicable to the vsftpd-3.0.2 version that was previously ...
8
votes
1
answer
1k
views
Why does Amazon Linux 2 have older version of wget vs. Amazon Linux 1?
We noticed that wget on Amazon Linux 2 is v1.14, but on the older version of Amazon Linux, it was v1.18.
Any idea why this would be? I'm fairly certain we installed from yum in both cases.
Is there ...
0
votes
1
answer
1k
views
Git clone, wget, ping not working on GCP server all of a sudden
My Google Cloud Platform server is suddenly unable to download anything via wget, git clone, and unable to apt-get install anything.
--2019-05-07 14:48:02-- http://google.com/ Resolving google.com (...
-3
votes
2
answers
5k
views
Can I assign the special IP address when I use wget?
I have a multi-IP addresses Server, a main IP 102.1.1.1 and a C segment 103.1.1.0/24 IP addresses on it.
when I use
wget www.google.com
can I assign my special ip address in my Server like ping?
...
0
votes
0
answers
6k
views
CentOS 7: curl and wget not working
My problem is that curl and wget usually fail to retrieve more than just the first part of any given file, and often nothing at all. Both have worked previously on the same server, and nothing else on ...
1
vote
1
answer
6k
views
Upgrading wget on Ubuntu 16.04 server
I'm trying to upgrade my copy of wget on the server. Currently its running:
1.17.1-1ubuntu1.4
The problem is that I need 1.19 so I can get a new feature:
http://tomszilagyi.github.io/2017/02/Wget-...
18
votes
6
answers
15k
views
Why the discrepancy between Speedtest and Wget?
My customer complains about low internet speeds. When measured with Speedtest.net speeds are acceptable. Periodic measured downloads are 10% to 30% of the nominal speed. I cannot explain that.
Some ...