2

I've recently taken over administration of a decent size network (900 FreeBSD or Linux hosts across 10 physical sites) where we get a lot of email destined for root, mostly from cfengine (v2). We've also installed a lot of cron jobs over the years, some of which make noise from time to time (and some of which make noise on every run!) Over the years, sysadmins on the team have grown to accept the messages and filter them out to /dev/null so it rarely gets read.

Obviously this isn't ideal and we should fix all the noise - and we will. But first I'd like to do something to wrap our heads around the problem, and try to point to the most "noisy" programs, and to keep stats on the volume of emails so that we can see stats on the changes we make.

I was thinking I'd set up a machine which receives copies of these emails and then process them into a database, where I'd run stats on how many repeats of similar messages we get. There are a lot of pre-existing systems out there for aggregating traditional log files, but the only one that similar to what we're after is Sentry, but I would have to write some processing stuff to make it work. I could write something myself if nothing like this exists.

Has anyone else faced this sort of issue and is aware of a solution for aggregating lots of error email reports and performing analysis on them? I'm looking for:

  • Stats over time on # of emails per hour or day
  • Lists of most common messages, by message content, subject or From: address
  • Any other stats which would point to where simple changes could have the most impact on volume of messages
1

I've had to cope with a very similar issue.

In my case, I was getting different emails from CFEngine, depending on whether they came from a cf-agent run from cron, or were run from cf-exec.d

What I ended up doing was removing the cron cf-agent call, and replacing it with a script that simply ensures that the cfengine3 service is running.

Now all the email comes from cf-execd, which makes my email rules much nicer. Its not quite what you're after, but it does make my day more manageable.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.