Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

How do I write a script that sends a get request to a web page and returns nicely formatted http headers?

share|improve this question
    
Platform, scripting language, any other detail you'd like to give? –  sparks May 21 '09 at 22:09
add comment

6 Answers

up vote 1 down vote accepted

try this nasty hack:

wget -O - -o /dev/null --save-headers http://google.com | \
awk 'BEGIN{skip=0}{ if (($0)=="\r") {skip=1;}; if (skip==0) print $0 }'
share|improve this answer
add comment

There are quite a few modules in many different languages that will retrieve HTTP headers for you.

etc, etc.

share|improve this answer
add comment

One option might be to use curl with the --dump-header option.

share|improve this answer
add comment

If you just want to view the headers.. (Not programatically) just use [live http headers][1] plugin for mozilla firefox.

https://addons.mozilla.org/en-US/firefox/addon/3829

share|improve this answer
1  
Now that firefox is mentioned, I just have to recommend FireBug. Perfect for analyzing what happens on a page - including network traffic. getfirebug.com –  Commander Keen May 22 '09 at 10:40
    
This is indeed what I use now - Firebug has excellent header tools. But they are a pain in the ass to copy and paste from, Firbug is not super-stable, and you can't script it to automatically check headers on multiple files. –  deadprogrammer May 22 '09 at 15:43
add comment

curl has support to view the headers when you're downloading, or you can use the -I option to save the headers to a file.

share|improve this answer
add comment

This bash function will take an URL + method, or Server + path + method + port. If you use "HEAD" method, it will return the headers, if you use GET, it will return all headers and the whole reply. Supports https through openssl.

#!/bin/bash
function httpreq ()
{
    if [ $# -eq 0 ]; then echo -e "httpreq SERVER PATH [GET/HEAD] [PORT]\nOR\nhttpreq URL [GET/HEAD]"; return 1; fi
    if echo $1 | grep -q "://"
    then
        SUNUCU=$(echo $1 |cut -d '/' -f3 | cut -d':' -f1)
        YOL=/$(echo $1 |cut -d '/' -f4-)
        PROTO=$(echo $1 |cut -d '/' -f1)
        METHOD=${2:-GET}
        PORT=$(echo $1| sed -n 's&^.*://.*:\([[:digit:]]*\)/.*&\1&p')
        if [ -z $PORT ]
        then
            if [ $PROTO == "https:" ]; then PORT=443; else PORT=80; fi
        fi
    else
        SUNUCU=$1
        YOL=$2
        METHOD=${3:-GET}
        PORT=${4:-80}
    fi
    if [ $PROTO == "https:" ];
    then
     echo -e "$METHOD $YOL HTTP/1.1\r\nHOST:$SUNUCU\r\n\r\n" | openssl s_client -quiet -connect $SUNUCU:$PORT
    else
     echo -e "$METHOD $YOL HTTP/1.1\r\nHOST:$SUNUCU\r\n\r\n" | nc $SUNUCU $PORT
    fi
}
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.