0

I have a mod_security log parsing script to load data into a mysql database which is then accessed by our Project Management System to display recent flaggings by the rulesets for our developers.

The script works great, but I need to work out how it's called. I could add it to a cronjob but I am not sure about server load running it every 2-5 seconds.

I was wondering if there is any way to have the script called when the log file is written to?

1 Answer 1

2

If you are writing to a file, you could just write to "|yourimportscript" and have it run off the piped input. When apache is restarted, it would restart the process. It would spend the majority of its time waiting until it received input and wouldn't require a cron job.

You could write your script as a daemon and use famd to see if the file was modified. Or, save the last read fpos, and when your cron fires, see if there is anything beyond the saved fpos.

If you aren't getting that much data, the impact would be somewhat low. If you are getting a lot of data, you could also consider logging via syslog to a separate machine and doing the processing there.

4
  • I don't actually have control of the write process. Mod_Security is doing the logging. Moving processing to a remote machine is not likely to help much as we are using a Virtualized solution. Famd is interesting I'll look into that. Jul 2, 2010 at 16:29
  • 1
    mod_security can be configured to write to a piped process with SecAuditLog "|pathtoyourprocess". You can also add a secondary log with SecAuditLog2 so that it can write to your existing log, and Log2 will write to your piped process.
    – karmawhore
    Jul 2, 2010 at 16:35
  • SecAuditLog |/var/log/modsec modsec is my script. In the script should I be looping through <STDIN> where I would be looping through a file? Jul 2, 2010 at 16:50
  • 1
    correct. It will only get data when mod_security writes to it. To test your script, grab a section of the logs and do /var/log/modsec < logsnippet
    – karmawhore
    Jul 2, 2010 at 16:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .