Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We want to store logiles so that they are easily searchable and available for extended periods of time. The problem is that there are multiple servers that needs to be archived.

I've looked in to a few database based solutions. Database is pretty much the only way of storing this data so that it can be analyzed and searched. The problem is that the databases go one of 2 ways. Most solutions only focus on storing the data, which can be seen as a lack of indexes. The indexes are easy to create, but by creating indexes the database starts to grow out of control.

What I'm looking for is a solution that offers a balance between these extremes. I want to have a relatively good searching capabilities (searching for whole words or IP addresses) while the database stays relatively small.

For a good measure, I've seen a database with ~4GB of data grow in to 40GB just because of the indexes.

share|improve this question

Splunk might just be what you're looking for. If you like FOSS better and don't need all the features of Splunk, take a look at graylog2.

share|improve this answer
I've run in to splunk before, didn't try it back then though. Graylog2 looks more like the app I want though. I'll need to take a look at these in a production environment if they behave in a way that we need. – Ressu Sep 22 '10 at 11:14

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.