Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a load balanced farm of servers that need to log various messages to an audit log, which will be processed off line at a later stage (This is a medical app, so all access to patient records must be audited). Currently, logs are being stored as XML using NTEXT data type to a SQL Server database, but I am looking for a more performant solution. Don't really need SQL because the logs are not queried by the servers, only processed at a later time.

I need a fast, high availability solution. I've looked into Facebook's Scribe. Can anyone suggest other alternatives? The servers are WCF services.

Thanks!

share|improve this question

4 Answers 4

up vote 1 down vote accepted

How about graylog2 ? It uses MongoDB as data storage.

http://www.graylog2.org/

share|improve this answer

You might have a look at Splunk for a software-based solution. Q1 Labs also makes hardware-based solutions.

Although I have to admit I'm a bit surprised that the way you're currently doing things isn't able to keep up.

It might be helpful to be more specific about your needs (i.e.: # of potential transactions, additional log file types, etc.)

share|improve this answer
    
Good point, GregD. Currently, we don't have any numbers on server load or log messages per second, but we are about to increase our log throughput by an order of magnitude, and we are starting to avoid logging because of fear of server overload. So, I am looking for a solution that scales well. Unfortunately, SQL Server with datbaase miroring (our current setup) doesn't scale out. –  Jacko Aug 2 '10 at 17:02

Depending on your needs syslog-ng might be what you want. Setup a central logging server (or many of them), and make your farm of servers log towards your logging server. Then just run the log analyzer at the logging server whenever necessary without affecting your actual servers at all.

syslog-ng is also able to receive syslog events from Windows servers, you just need to install a small service to Windows. I don't remember the name straight away, though, since I'm not normally a Windows user.

share|improve this answer

I've decided to look into Flume (http://github.com/cloudera/flume). If I can build it on windows. It provides guaranteed delivery, which is important to me. Unlike Scribe.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.