File based log monitoring with triggers for mails ONLY?

Question

I have various heterogenous services from my company able to log things into files, but e.g. not able to send mails on their own. Additionally, I have logging into files from Apache httpd and Tomcat, Postgres, Upstart and various other sources by default not using e.g. Syslog. The commons sense of all those services is using files, but log size, formats of the messages, rotation or not etc. differ.

So what I'm looking for is some log monitor which focusses on configured files and provides triggers to match newly added content to those files against some regular expressions or such to send mails with error messages.

I would like to have something stand-alone, focussing on log file monitoring with triggers and being somewhat efficient by e.g. using file system events to get notified of changed files, only reading the end of files on changes instead of parsing the whole file and such. I need some flexibility in defining triggers, though, but besides that it only needs to send mails with log messages to some arbitrary addresses. Not even on its own, supporting a local sendmail is absolutely enough, with very little templating features for those mails. I don't even care much for things like recognizing multiple error messages or such.

I've read the whole day about that topic and found various different solutions, things like Graylog, Logstash, plugins for Nagios/Zabbix etc. But from what I've read, all of those have serious drawbacks for my environment: Graylog and Logstash seem to have a heavy setup and dependencies and provide functions I simply don't need, like searching in logs. I currently don't use Nagios and Zabbix either and am not sure how their log monitoring plugins perform, because they seem to poll log files time based and such. Additionally, Zabbix's config looks a bit hard to maintain as well.

So is there something more focussed available?

Answer 1

I strongly suggest you to learn and implement a full-blown monitoring solution as Zabbix: while you only need log file monitoring today, you may need something more comprehensive tomorrow.

If you really only want to monitor some log files, give a look at the answers that you can find here and here.

Answer 2

Monit sounds promising, as per @shodanshok's hint to other questions:

check file syslog with path /var/log/syslog
    ignore content = "monit"
    if content = "^mrcoffee" then alert

SwatchDog seems interesting, but old and unmaintained, with serious open bugs.

logster seems interesting, but Python is not my favourite choice and it seems to focus on providing special output instead of trigger actions. Might have serious bugs as well.