Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am trying to setup a log monitoring application for monitoring Jboss Application logs. We have more than 50 Jboss Application servers in all our environments. My Requirement is below.

  1. The log monitoring app should be open source.
  2. There is less configuration required in client (Jboss) for fetching the log.
  3. The app should capable of sending alerts to mail group.
  4. It should be capable of detecting duplicate alerts.
  5. We don't need to any kind of search or indexing of log data.
  6. And also I want to filter Known Errors in the log, So that we dont get alerts for such errors.

We tried zabbix log monitoring but it has a lack of intelligence for detecting duplicate alerts.

After lot of Google search I heard about Sentry. I had configured the server successfully. But I don't know it is possible to monitor my Jboss application log with Sentry.?

Do anyone have experience in monitoring Jboss Application logs using Sentry?

Or there is any better open-source log monitoring app available which covers all my requirements above?

share|improve this question
    
What do you mean by "detecting duplicate alerts"? –  quanta Mar 13 '13 at 9:34
    
For example, if you happen to receive 100,000 events/second, and 10% of those are reporting a connection issue to the database (where they’d get grouped together). And the Monitoring app capable realizing that all those are same kind of error and only send 1 alert saying that "DB connection timeout occurred in 1000 times". Ref : sentry.readthedocs.org/en/latest/buffer/index.html –  ArunS Mar 13 '13 at 9:43
    
-------------LOG------------------ 2013-03-13 10:48:56,442 ERROR [nucleusNamespace.app.ecommerce.search.QueryFormHandler] (ajp-0.0.0.0-8239-71) beforeSearch() can not execute paged request with no previously saved request -------------LOG------------------ I want to get all occurrence in the log except DEBUG and INFO. –  ArunS Mar 13 '13 at 9:51

1 Answer 1

There are lots of open source log monitoring tools out there. the problem with most of them is, as you noticed with zabbix (although that's not free), they lack intelligence. those that do have any type of intelligence are limited in what they can do.

I monitor approximately 1100 unix hosts, and having a reliable and efficient way to alert on the log files on those hosts has proven very helpful. I use Nagios for this. Nagios has a plugin called LOGROBOT. This plugin can be used with or without Nagios but I use it with Nagios because I love Nagios.

Let's do a quick run down on your list of requirements:

1. The log monitoring app should be open source.

(Nagios is free to use)

2. There is less configuration required in client (Jboss) for fetching the log.

(if using LOGROBOT with Nagios, any alert configuration you wish to do on the remote logs can be done from the Nagios host - no need to have to ssh into the servers)

3. The app should be capable of sending alerts to mail group.

(Nagios automatically does this, you can specify as many email addresses as you wish on the Nagios host)

4. It should be capable of detecting duplicate alerts.

(Nagioslogrobot sends an alert based on what you want to alert on. it has its own algorithm it uses to know when to alert as PROBLEM or as a RECOVERY. you will only receive alert notifications on real issues)*

5. We don't need to any kind of search or indexing of log data.

(Nagios does not do any indexing/searching of log data - that wouldn't fall under log monitoring anyway)

6. And also I want to filter Known Errors in the log, So that we dont get alerts for such errors.

(Nagios and logrobot allows for the specification of a list of strings/patterns to exclude).

There are several examples I can give here, but i dont want to make this too long. To directly address your concern for being able to exclude strings, try this:

Command:

NagiosLogMonitor remotehost logrobot autonda /var/log/app.log 60m 'DB Timeout' '(DEBUG|INFO)' 5 10 dblogcheck -ndshowexcl

a) The above command will connect into remotehost

b) It will check the log file /var/log/app.log

c) It will abort if the log file has not been written to in over 60 minutes

d) If update time of log file is less than 60 minutes, it will search for all lines in the log that contain "DB Timeout".

e) If it finds any line containing "DB Timeout", it will remove from those lines all lines that contain either DEBUG or INFO.

f) If the number of lines left is at least 5, but less than 10, it will alert and send a WARNING email to you. this email will contain the lines it found.

g) If the number of lines left is 10 or over, it will alert and send a CRITICAL email to you. this email will also contain the lines it found.

I may be forgetting something here, but that should cover it.

share|improve this answer
    
Logrobot is not open source. But that was a requirement by the one asking the question. –  guettli Apr 29 '13 at 14:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.