Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am not a system administrator but I run a little website and today I have received a subpoena request from police, in order to comply I need to extract IP addresses from our server logs for needed time.

This is my problem:


I have log1.txt

/var/log/lighttpd/access.log.1.gz: - [28/Apr/2011:09:23:30 -0500] "POST /base/script.php HTTP/1.1" 200 158 "http://ref/,http://ref/" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/ Safari/530.5"
/var/log/lighttpd/access.log.1.gz: - [28/Apr/2011:09:23:30 -0500] "POST /base/script.php HTTP/1.1" 200 158 "http://ref/,http://ref/" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/ Safari/530.5"

The file contains 500+ such lines

This is the output I am expecting:,, and other ips.


And I have log2.txt that has exactly same format

I need to output IPs from that log as well, e.g.,,


I need to use output results from log1 and match them with log2 and if one IP repeats in both logs then output it. From the examples I have provided, that IP would be

Could you assist?

share|improve this question

I'd suggest using cut, not grep:

cut -d\ -f1 log1.txt | sort | uniq > ip1.txt
cut -d\ -f1 log2.txt | sort | uniq > ip2.txt
grep -f ip2.txt ip1.txt

If the IP you're after is the second in each line, rather than the first, replace '-f1' with '-f2'.


share|improve this answer
Thanks, but it doesn't work: cut: the delimiter must be a single character – Andrey Apr 30 '11 at 8:59
There's two spaces after the backslash; the first is the delimeter. Perhaps -d' ' instead? – caelyx Apr 30 '11 at 9:00
Thank you. Seems to work now. – Andrey Apr 30 '11 at 9:01
Thank you, but the output is not what I need. /var/log/lighttpd/access.log.1.gz: How to make it to output instead. – Andrey Apr 30 '11 at 9:02
Oh, I gave you the wrong file format. All lines also begin with /var/log/lighttpd/access.log.*.gz:, e.g. /var/log/lighttpd/access.log.1.gz: or /var/log/lighttpd/access.log.2.gz: How to modify the cut query to work with this? Thank you! Let me also edit the original post. – Andrey Apr 30 '11 at 9:05

If all you want is a list of ip addresses

cut -d ' ' -f1 log1.txt | cut -d ':' -f2 | sort | uniq >log1.out
cut -d ' ' -f1 log2.txt | cut -d ':' -f2 | sort | uniq >log2.out
while read IP
    sed -n /$IP/p log2.out
done <log1.out
share|improve this answer
I gave wrong file format in the original post. I have now modified it. – Andrey Apr 30 '11 at 9:07
@Andrey: I updated my answer – Iain Apr 30 '11 at 9:19
Thank you Lain and Caelyx. I have accomplished my desired objective. – Andrey Apr 30 '11 at 9:27

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.