Ok, a little back story first,
We have a Tomcat instance that connects to multiple service providers as a sort of "gateway." This Tomcat instance will generate errors if it cannot connect to a given provider. These errors are stored in a log file.
The problem is that we need to monitor the file for more than just a given string, we need to use several regular expressions to filter through all the meaningless stuff so that we aren't waking up at 3am because a user forgot his password. ;)
I already tried creating a python script that would sit between Zabbix and Tomcat, but with log files (that are rotated daily) that are in excess of 200MB, it's too CPU intensive (100% for 15+ seconds). I tried adding a "sleep(0.000005)" to slow it down a bit, but now it takes to long (> 1min) and still uses 25% CPU...
All in all, the solution is overshadowing the problem and I wanted to reach out to you guys and see if you know of a better solution, or anything else that might help.