Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have an app which simply logs to a file. I am planning to deploy this app on cloud. If I deploy it on AmazonEC2 instances (multiple), I would want to monitor logs of every instance. Something which shows the logs in a structured manner and alerts if anything grave is logged.

any ideas ?

ps : if this is a repeat question, feel free to link that question here

share|improve this question
up vote 1 down vote accepted

I'm using syslog-ng for that, you can configure syslog-ng on your local machine to read from the destination file (or you can make your app log to syslog) and then send it to a destination machine where all the logs are either merged into one or divided in different files and even a directory structure.

On the client side

source yourservice {
    file("/var/log/xxx.log" follow_freq(1) flags(no-parse));

destination yourcollector {
    tcp("xx.xx.xx.xx" port(65140));

log { source(yourservice); destination(yourcollector); };

Then on the collector server

source yourservice {
    tcp(port(65140) keep-alive(yes));

destination yourservice_dir { file("/var/log/yourservice/$YEAR/$MONTH/yourlog-$YEAR-$MONTH-$DAY" create_dirs(yes) template("$MESSAGE\n") template_escape(no));};

log { source(yourservice); destination(yourservice_dir); };

You can change the parameters on the destination to add or remove variables to divide it in one or more logs

share|improve this answer
let me try that out :) seems good now :) – Shrinath Feb 10 '11 at 3:43

protected by Michael Hampton Oct 27 '14 at 18:23

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.