We have about a thousand servers / virtual machines, and currently keeping track of who did what where isn't easy. And to make it worse, clients have access to their own machines and often don't know themselves who may have done what with their access.
I'm looking for :
- Log every command ran by users and root
- Have a way to tell SSH sessions from cron or other ways of running a command (PHP ..)
- Log the env, or at least one env variable for every command (to tell apart multiple users logged in as root)
- Have the time the command ran ? To try and know how long vim was opened on a particular file to correlate with service reloads ran in another shell for example
- Ideally there'd be no way of bypassing the logging
I don't mind developing my own set of tools to parse the big quantities of log into something usable, I'm mostly looking for the best way to get the information itself.
Reading this thread it seems like auditd was the best way of doing it in 2013, is it still the case ? I imagine it wouldn't log shell builtins like echo, but maybe that's not a huge deal. Is there a way to configure it to tell you what process spawned what (to tell apart sshd from cron, PHP ..) ? Is there a way to log an env variable for each execve ?
It looks like there are much easier tools available, but they're either heavy (snoopy ?) or easy to bypass, which makes it a bit pointless. Servers hosting wordpress for example are often "hacked" and it'd be nice to have a log of what exactly happened when we detect it later.