Is there a way to tail a resource such as http://someserver.com/logs/server.log ?
This is what I would do for a local file.
tail -F /var/logs/somefile.log
I would like something similar to that for a file accessed through the http protocol
Server Fault is a question and answer site for system and network administrators. It only takes a minute to sign up.Sign up to join this community
You might use the script from https://stackoverflow.com/a/1102346/401005 if you are able to run php scripts. You should add a
flush(); after the
curl --no-buffer http://the/url you should have an suitable output
This will do it:
#!/bin/bash file=$(mktemp) trap 'rm $file' EXIT (while true; do # shellcheck disable=SC2094 curl --fail -r "$(stat -c %s "$file")"- "$1" >> "$file" done) & pid=$! trap 'kill $pid; rm $file' EXIT tail -f "$file"
It's not very friendly on teh web-server. You could replace the
sleep 1 to be less resource intensive.
tail -f, you need to
^C when you are done watching the output, even when the output is done.
htail (https://github.com/vpelletier/htail) and it seems to do the job pretty well.
If you want a sentinel program type like I needed ( only update when you respond )
#!/bin/bash while true; do echo "y to curl, n to quit" read -rsn1 input if [ "$input" = "y" ]; then curl $1 elif [ "$input" = "n" ]; then break fi done
18:02:44] $: ~ \> curlTail.sh http://wherever y to curl, n to quit
When you hit y, it'll curl it, then prompt you again.
I wrote a simple bash script to fetch URL content each 2 seconds and compare with local file
output.txt then append the diff to the same file
I wanted to stream AWS amplify logs in my Jenkins pipeline
while true; do comm -13 --output-delimiter="" <(cat output.txt) <(curl -s "$URL") >> output.txt; sleep 2; done
don't forget to create empty file
output.txt file first
: > output.txt
view the stream :
tail -f output.txt
I found better solution using wget here:
while true; do wget -ca -o /dev/null -O output.txt "$URL"; sleep 2; done