Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a traffic log similar to this

"page visited" for xxx.xxx.xxx.xxx at 2013-10-30

and I would like to replace the ip address with the result of it's whois lookup.

I can export the ip addresses to a separate file and then do a whois on each line, but im struggling to combine them all together.

Ideally i'd like to replace the ip address in the same string and print the new string to a new file.

So it would look like

"page visited" for example.com at 2013-10-30

Can anyone help

Here's what I have so far

grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' clean_cites.txt > iplist.txt
for i in `cat iplist.txt`
do
OUTPUT=$(geoiplookup -f /usr/share/GeoIP/GeoIPOrg.dat $i)
echo $i,$OUTPUT >> visited.txt
done

Like I said,this produces a separate file with a list of ip addresses and their relevant hostnames, so I either need to search for the ip address in file and and replace it with the text in file b (which will give the ip address and hostname) or replace the ip address in place.

Thanks

share|improve this question
    
Can you post your script so far? –  sgtbeano Nov 11 '13 at 12:41
2  
what if there's more than one DNS name? –  Chopper3 Nov 11 '13 at 12:43
    
Not exactly related to solving your problem. but what does your geoiplookup output? GeoIP databases usually only provide geographic location information which doesn't appear to be what you're after. The normal method used by most software to resolve IP addresses is to get reverse DNS with something like dig +short -x a.b.c.d. Not all IP addresses have reverse DNS though. –  USD Matt Nov 11 '13 at 12:55
    
It outputs the name of who the ip address belongs to...we've purchased the extra database. –  user2099762 Nov 11 '13 at 12:59
    
For clarity - there is an edition of MaxMind's GeoIP Database that supports domain resolution. Is this what you're using? –  plasmid87 Nov 11 '13 at 14:01
add comment

1 Answer

You don't need to save the IP addresses to a temporary file. You don't even need to use grep since Bash 3.2 and greater supports regexes (3.1 also does, but the features were changed in 3.2).

#!/bin/bash
ip_pattern='([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}'
while read -r line
do
    [[ line =~ $ip_pattern ]]
    ip=${BASH_REMATCH[0]}
    domain=$(geoiplookup -f /usr/share/GeoIP/GeoIPOrg.dat "$ip")
    echo "${line//$ip/$domain}" # substitute the domain for the ip in the original line
done < clean_cites.txt > outputfile.txt

This should run faster since you don't have to spawn grep for every line and you're not having to iterate over the file twice (or two or more files).

You should never do for i in cat by the way, use while read -r instead.

Make it a habit to use mixed case or lower case variable names in order to avoid potential name collision with shell variables.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.