Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an apache / nginx whatever webserver which logs client ip addresses to the access logs. Now these log files are rotated via logrotate.

I want to keep the IP addresses for some days, then - after 7 days, i want to remove the ips from the log files for privacy reasons (mostly dictated by german law)

using mod_removeip or something like that doesn't work because I need to filter some requests based on their IP addresses.

Is there any 'standard' way how to do that - maybe even with logrotate?

EDIT

i just found this script http://www.zendas.de/technik/sicherheit/apache/index.html but it depends on the ability to pipe all logging trough the script in real-time. i'm not really sure about the performance implication of this approach.

Also, this only works for the 'front-end' server logs, not the application server logs

share|improve this question
add comment

2 Answers 2

I don't think logrotate will do it; you may need to look at creating a script that will decompress the files, process them through awk or sed to strip the IP's out, then recompress them. Just can't do it on "active" log files.

share|improve this answer
3  
I believe logrotate has pre/post hooks that you could use to launch the script you mention, then the OP wouldn't need to manage a separate process. –  EEAA Feb 9 '12 at 14:17
3  
Maybe you can use logroate's "postrotate" for this. –  Stone Feb 9 '12 at 14:21
    
i even thought of creating a "compress" script which filters and then pipes to gzip. this would essentially save the step of decompressing the logs but would 'kill' the time window of 7 days i want –  Michael Siebert Feb 9 '12 at 14:31
add comment

PCRE! (Perl-Compatible Regular Expression)

s/\b(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\b/REMOVED IP/g

Use that as a filter in a perl script or any other suitable language (quite a few use PCRE or some other close-enough regex language that will work) to rewrite your log files at 7 days.

$ cat > file_with_ip
some text from 192.168.1.1
^D
$ perl -p -i -e 's/\b(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\b/REMOVED IP/g' file_with_ip
$ cat file_with_ip
some text from REMOVED IP
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.