Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am not a system administrator but I run a little website and today I have received a subpoena request from police, in order to comply I need to extract IP addresses from our server logs for needed time.

This is my problem:

1

I have log1.txt

/var/log/lighttpd/access.log.1.gz:84.20.132.141 180.175.44.143 - [28/Apr/2011:09:23:30 -0500] "POST /base/script.php HTTP/1.1" 200 158 "http://ref/,http://ref/" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.28 Safari/530.5"
/var/log/lighttpd/access.log.1.gz:85.40.142.111 180.175.44.143 - [28/Apr/2011:09:23:30 -0500] "POST /base/script.php HTTP/1.1" 200 158 "http://ref/,http://ref/" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.28 Safari/530.5"

The file contains 500+ such lines

This is the output I am expecting: 85.40.142.111, 84.20.132.141, and other ips.

2

And I have log2.txt that has exactly same format

I need to output IPs from that log as well, e.g. 85.40.142.111, 81.02.32.61,

3

I need to use output results from log1 and match them with log2 and if one IP repeats in both logs then output it. From the examples I have provided, that IP would be 85.40.142.111.

Could you assist?

share|improve this question

2 Answers 2

I'd suggest using cut, not grep:

cut -d\ -f1 log1.txt | sort | uniq > ip1.txt
cut -d\ -f1 log2.txt | sort | uniq > ip2.txt
grep -f ip2.txt ip1.txt

If the IP you're after is the second in each line, rather than the first, replace '-f1' with '-f2'.

HTH.

share|improve this answer
    
Thanks, but it doesn't work: cut: the delimiter must be a single character –  Andrey Apr 30 '11 at 8:59
    
There's two spaces after the backslash; the first is the delimeter. Perhaps -d' ' instead? –  caelyx Apr 30 '11 at 9:00
    
Thank you. Seems to work now. –  Andrey Apr 30 '11 at 9:01
    
Thank you, but the output is not what I need. /var/log/lighttpd/access.log.1.gz:93.8.158.46. How to make it to output 93.8.158.46 instead. –  Andrey Apr 30 '11 at 9:02
    
Oh, I gave you the wrong file format. All lines also begin with /var/log/lighttpd/access.log.*.gz:, e.g. /var/log/lighttpd/access.log.1.gz: or /var/log/lighttpd/access.log.2.gz: How to modify the cut query to work with this? Thank you! Let me also edit the original post. –  Andrey Apr 30 '11 at 9:05

If all you want is a list of ip addresses

#!/bin/bash
cut -d ' ' -f1 log1.txt | cut -d ':' -f2 | sort | uniq >log1.out
cut -d ' ' -f1 log2.txt | cut -d ':' -f2 | sort | uniq >log2.out
while read IP
do
    sed -n /$IP/p log2.out
done <log1.out
share|improve this answer
    
I gave wrong file format in the original post. I have now modified it. –  Andrey Apr 30 '11 at 9:07
    
@Andrey: I updated my answer –  Iain Apr 30 '11 at 9:19
    
Thank you Lain and Caelyx. I have accomplished my desired objective. –  Andrey Apr 30 '11 at 9:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.