I am not trying to create an opinionated question here though it might sound like it. My problem is as follows.

I have the normal apache2 access and error log on a bunch of servers. I also have custom logs with custom formats.

I have noticed a beginning performance issue due to the amount of lines being written into the log files so I am looking for alternatives.

I would love to somehow gather the log entries. I have been trying to understand how syslog works and I think that's the direction I need to go since the syslog would function as a kind of buffer.

I know of a program called greylog... But I am not sure if that is the direction I should go... Maybe someone have experience on a similar setup and could share how they made it work?

  • Have you considered logrotate? – Alexey Ten Apr 23 '14 at 9:19
  • 1
    I am rotating the log via log rotate.. but that doesn't change the issue with a lot of IO... I need some other way of storing the logs... – Ronnie Jespersen Apr 23 '14 at 9:21

Have you tried to log using syslog with Apache's CustomLog directive?

For example,

CustomLog "|/usr/bin/logger -t 'httpd'" combined

That might help you performance-wise, especially if you tune the syslog daemon itself to write asynchronously. With more fine-grained sysloggers such as syslog-ng, you have many performance-related tunables.

Another advantage of this approach would be that this way you could send everything to a centralized log server and let it receive the disk I/O strain instead of your web server.

  • No I haven't. I would definitely like to send it to another server for storage. How would I tell the logger to send to another server? And how would the server receive the log entry? – Ronnie Jespersen Apr 23 '14 at 11:28
  • logger is just a messenger, the real "please send this log line to another server" work is done by the syslog daemon your server has. For example, with the traditional syslogd, you would send logs to another server with @someserver parameter. The centralized log server then receives the message and puts it in the directory & file it seems best. syslog-ng can be kind of plug-and-play solution for your centralized logging; with its templates you can define that logs will be stored in /var/remote_logs/$hostname/ fashion so you could find your logs per-host basis. :) – Janne Pikkarainen Apr 23 '14 at 11:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.