Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Once a day, I want to run AWStats on webserver log files generated by multiple load balanced servers. I want an efficient way of transfering them to one place. Is there already a tool that can do this?

Otherwise, I was thinking of using a cron job to grep for the current day, then tar and gzip the files before sending them over so I can merge and analyze them. Is this a good approach or can you suggest a better approach?

Thanks!

share|improve this question
add comment

5 Answers

up vote 2 down vote accepted

There are tools that are meant to be a Central Log Management System for Linux, that doesn't require them to be copied. At least, not in the sense you are talking about, you can just set up NFS mounts or install clients on the machines.

An easy one with a nice web interface is Splunk, it is free for up to 500MB a day of indexing without authentication for the web interface.

The classic more manual method is syslog-ng which might already be on your system. Here is a tutorial on setting up a central log server with that.

share|improve this answer
add comment

Just rsync the logs over to your analysis machine, saves a hell of a lot of unnecessary logic.

share|improve this answer
add comment

Use rsync

I'd use rsync in a cron job. Fast, reliable, simple.

share|improve this answer
add comment

In AWstats tools you can find a perl script (logresolvemerge.pl) for merge log files in case of load balancing. This script can help you AFTER the copy (rsync is good choice - see answeres of jodiek and KPWINC) of web server's logs.

share|improve this answer
add comment

Have you considered using NFS on the servers to mount a directory from another server?

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.