For some reason, some script or program is removing a pid file for the service on the linux server (centos5.4 / 2.6.18-308.4.1.el5xen). I suspect a faulty cron script, but manual investigation did not lead me to it. And i still want to track it down. Have been using auditctl rule:
auditctl -w /var/run/some_service.pid -p w
Which helped me to see something, but not quite exactly what i wanted:
type=PATH msg=audit(11/12/2013 09:07:43.199:432577) : item=1 name=/var/run/some_service.pid inode=12419227 dev=fd:00 mode=file,644 ouid=root ogid=root rdev=00:00
type=SYSCALL msg=audit(11/12/2013 09:07:43.199:432577) : arch=x86_64 syscall=unlink success=yes exit=0 a0=7fff7dd46dd0 a1=1 a2=2 a3=127feb90 items=2 ppid=3454 pid=6227 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts0 ses=38138 comm=rm exe=/bin/rm key=(null)
Problem here is that i see ppid of the script that removed the file, but at the analysis time the (p)pids are already invalid as probably scripts/programs have been shutdown. Imagine a cron script deleting the file. So i need some way to expand/add audit rule(s) to be able to trace the parents of the /bin/rm at the time of removal.
I have been thinking to add some rule to monitor all process creation, something like:
auditctl -a task,always
But this happen to be very resource intensive. So i need help or advice how to combine these rules, or how to expand any of the rules to help track the script/program. Thanks.