Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Right now, my organization has a solution comprised of 10+ components, and some have a log file per thread. Since files are rotated hourly, tracking all of this is a chore.

Is centralizing all logging to a specific machine (using rsyslog or something similar) a good idea? Wouldn't I be trading simplicity for hecticness? Are there good log viewers for this high volume use case?

We're a straight up Microsoft shop, by the way.

Thanks for any replies!

share|improve this question

5 Answers 5

up vote 3 down vote accepted

I would suggest you take a look at Splunk.

I currently have it in production, with 30+ network devices logging to it--It is really useful to have logs in one place, that I can write my own queries for, run canned reports, etc.

share|improve this answer
3  
It's bloody expensive though, especially when you get up to >5GB/day of data. –  Tom O'Connor Oct 6 '10 at 15:18
    
@ Tom: Definitely. –  Josh Brower Oct 6 '10 at 18:09
    
just how expensive? –  Bruno Antunes Oct 7 '10 at 10:50
4  
@sardaukar jaw dropping, are you f'in kidding me expensive –  Zypher Aug 4 '11 at 1:52
1  
@sardaukar Last pricing I got for 5GB/day was $32,000, and about $10,000 a year for support –  Owen Orwell Oct 11 '12 at 9:01

One BIG advantage of centralized logging is this:

  • If one of your machines is ever compromised, and the logs altered to hide that fact, you will still have an un-tampered copy on your central logging server.

Another is:

  • In my case I also have a dedicated monitor at my workstation running off of the central logging server that displays any logs of a priority "Warning" or higher in real time, so that I can deal with any problems immediately as they come up. (hopefully before the end-user notices :) ). This is difficult to do without a centralized server.
share|improve this answer

Take a look at eventsentry as well, not very expensive for few licenses, can setup good filters and alerting etc.

share|improve this answer

Centralized logging is always a good idea, provided your log server is in a secured location.

Have you looked at Splunk and/or Spiceworks?

www.splunk.com www.spiceworks.com

share|improve this answer
    
Spiceworks is not a logging app, so I am not sure what it would do for the OP(?) –  Josh Brower Oct 6 '10 at 13:30
1  
Spiceworks does have some basic log charting and checking features (community.spiceworks.com/help/Windows_Event_Logs) but nothing like dedicated log management software. Splunk is a good choice. –  Jeff Halley Oct 6 '10 at 14:21
    
I suggested both, because I use them together to monitor both my local and hosted environments. –  tagram Oct 6 '10 at 14:30

Our AD DC security logs go through between 3-5GB of logs a day during busy times, and there is just no way on this Earth to do anything with them via native tools. Some kind of log-parser is needed to make sense of them. I wrote one from scratch in PowerShell, and we've recently looked at Splunk. Splunk can keep up to the deluge and also keep up with our net-device syslog data (almost as big volume wise). All in one database. It'll take a beefy server to chew through that kind of data load, but that's a solvable problem. We're currently waiting on the right Dark Rites to complete so we can get funding for a centralized logging environment.

Having a "single pane of glass" as a view into the data is a good thing. What you won't get is an updating text-file you can then tail syslog-style. What you will get is an interface with a rich query system, and (I believe) an API for writing your own web front-ends for your own nefarious needs.

When it comes to Windows Event-log data, Splunk is not pulling out the XML version of those events, it's pulling out the "detailed view" text version of each event and parsing that. I had some real worries about scale there, but I was pleasantly surprised when it managed to keep up to our logging loads in spite of that; I had to go XML with my PowerShell script because text-parsing was taking w-a-y too long.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.