Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.
  • Ubuntu 10.04.2
  • nginx 0.7.65

I see some weird HTTP requests coming to my nginx server.

To better understand what is going on, I want to dump whole HTTP request data for such queries. (I.e. dump all request headers and body somewhere I can read them.)

Can I do this with nginx? Alternatively, is there some HTTP server that allows me to do this out of the box, to which I can proxy these requests by the means of nginx?

Update: Note that this box has a bunch of normal traffic, and I would like to avoid capturing all of it on low level (say, with tcpdump) and filtering it out later.

I think it would be much easier to filter good traffic first in a rewrite rule (fortunately I can write one quite easily in this case), and then deal with bogus traffic only.

And I do not want to channel bogus traffic to another box just to be able to capture it there with tcpdump.

Update 2: To give a bit more details, bogus request have parameter named (say) foo in their GET query (the value of the parameter can differ). Good requests are guaranteed not to have this parameter ever.

If I can filter by this in tcpdump or ngrep somehow — no problem, I'll use these.

share|improve this question
    
Can you characterize/classify the requests that you consider "weird"? How could one come up with a rule to help you if you don't share with us what is 'normal' and 'bogus'? –  hobodave Mar 13 '11 at 2:41
    
I do not ask for a rule — I can write it easily myself. I ask for means to dump HTTP request data. –  Alexander Gladysh Mar 13 '11 at 12:14
    
@hobodave: but anyway, since you've asked, I've added this information to the question. –  Alexander Gladysh Mar 13 '11 at 12:32

3 Answers 3

up vote 13 down vote accepted

Adjust the number of pre/post lines (-B and -A args) as needed:

tcpdump -n -S -s 0 -A 'tcp dst port 80' | grep -B3 -A10 "GET /url"

This lets you get the HTTP requests you want, on the box, without generating a huge PCAP file that you have to offload somewhere else.

Keep in mind, that the BPF filter is never exact, if there are a large number of packets flowing through any box, BPF can and will drop packets.

share|improve this answer

If you proxy the requests to Apache with mod_php installed you can use the following PHP script to dump the requests:

<?php
$pid = getmypid();
$now = date('M d H:i:s');
$fp = fopen('/tmp/intrusion.log', 'a');

if (!function_exists('getallheaders')) 
{ 
    function getallheaders() 
    { 
           $headers = ''; 
       foreach ($_SERVER as $name => $value) 
       { 
           if (substr($name, 0, 5) == 'HTTP_') 
           { 
               $headers[str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5)))))] = $value; 
           } 
       } 
       return $headers; 
    } 
} 

function ulog ($str) {
    global $pid, $now, $fp;
    fwrite($fp, "$now $pid {$_SERVER['REMOTE_ADDR']} $str\n");
}

foreach (getallheaders() as $h => $v) {
    ulog("H $h: $v");
}
foreach ($_GET as $h => $v) {
    ulog("G $h: $v");
}
foreach ($_POST as $h => $v) {
    ulog("P $h: $v");
}
fclose($fp);

Note that since you're using nginx the $_SERVER['REMOTE_ADDR'] may be pointless. You'll have to pass the real IP to Apache via proxy_set_header X-Real-IP $remote_addr;, and you can use that instead (or just rely on it being logged via getallheaders()).

share|improve this answer
    
Thank you. But I do not have PHP on my servers. Still the idea is valid for any other http-aware programming laguage. :-) –  Alexander Gladysh Mar 13 '11 at 21:22

I don't know exactly what you mean with dump the request but you can use tcpdump and/or wireshark to analyze the data:

# tcpdump port 80 -s 0 -w capture.cap

And you can use wireshark to open the file and see the conversation between servers.

share|improve this answer
    
Thanks, but I have quite a bit of traffic on this server (99% of it is good), and I think that it would be hard to filter out that bunch of data for that bogus 1% I need. –  Alexander Gladysh Mar 13 '11 at 1:28
    
...if I capture all of it on such low level. :-) –  Alexander Gladysh Mar 13 '11 at 1:35
    
I've updated the question to reflect that. –  Alexander Gladysh Mar 13 '11 at 1:41
    
Alexander - well that means that 1 out of every 100 requests will have the weird headers you're looking for. Run a capture for a while and then search through the resulting log looking for the headers you want - that's surely not an unbearable amount of work. –  EEAA Mar 13 '11 at 1:49
    
The problem is not the work, but the amount of data to process. (It may be bearable though, but, anyway, I'd like to see a more friendly solution.) –  Alexander Gladysh Mar 13 '11 at 2:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.