5

I was wondering if there are any benefits or common use-cases in which using separate Apache logs (access, agent, error, and referer) is better than using a single combined log file. I tried Googling it, but it seems that an analysis has not really been done on this particular dilemma.

Traditionally I have used separate ones simply because it made sense to separate data into groups (of course in another way, it is actually splitting grouped data; i.e., any given access is being split into multiple pieces—which means viewing four files at the same time and trying to keep them synchronized to the same timestamp).

I haven’t used too many log analyzers, but I suspect that at least some work better (or at all) with combined logs.

So is there a good reason to use separate over combined?

Improve this question
2
  • 2
    If you're troubleshooting errors on a high-traffic site, it's going to be a lot easier to have a dedicated error log than trying to sift through a few million pageviews to find the errors.
    – ceejayoz
    Jan 30, 2013 at 15:19
  • Yes, a separate error log makes sense. But then it seems like a combined access/agent/referer log makes more sense than separating them.
    – Synetech
    Jan 30, 2013 at 18:25

2 Answers 2

Reset to default
2

This is really a personal call that you have to make based on your environment and the way you work. There are advantages and disadvantages to both.

The major advantage of a combined log (everything going to one file) is that it's written out contemporaneously: If an error occurs you see it right under the "access" line (the request) that caused it.
This can make troubleshooting a little easier if you don't have a huge volume of requests.

Conversely, separate log files makes sense whenever the volume of entries is so large as to be overwhelming when trying to troubleshoot a problem.

The "traditional" division for Apache is access and error logs --

  • The Access log contains all the requests the server received (and the response codes it sent back), and sometimes additional useful bits like SSL Protocol, etc..
  • The Error log contains, as its name implies, errors (CGI/PHP/etc. failures, exceptions, stack prints, debugging output).

When you're hunting down a problem you watch the error log and provoke the problem, and you get just the errors (as opposed to the error plus all the GET requests for CSS files, Javascript, images, etc. that your browser went to download before you reached the part where things crash).

You can take this further to an extreme (separate Access, Agent, Referrer, SSL, etc. logs), but personally I see no need to do that. The traditional "Error" and "Access" (everything that isn't an error) logging is what you'll see in most deployments.

There may be a reason to keep additional dedicated logs for referrers, etc. if you're doing analysis or statistics, or you may omit that information from the Access log if you don't care about it, but I wouldn't subdivide logs to the point where you have to look in more than 2 places to troubleshoot a problem. That just winds up making more work for you.

Improve this answer
1
  • That makes sense. I thought 404’s and such went into the error log, but they don’t, they go in access.log, so it makes total sense to keep an error log and a separate Access+Agent+Referer log because then the access-related items are grouped together by time. (Personally I don’t recall ever having to see one of those individually, and in fact have had to coordinate the three logs to get a sense of any given access.) I don’t know if any log-analyzers choke on separate logs, but it won’t matter if using a combined one (plus, log-analyzers don’t need the error log anyway, only the access log).
    – Synetech
    Jan 31, 2013 at 15:27
1

I use different logs to separate key subjects which are running on the same server but are of totally different purpose (i.e. different sites or different type of security / sensitivity) as I find it easier to inspect, review and analyze as it cuts down the noise.

You could achieve similar results using grep too obviously.

Improve this answer
2
  • Do you mean using per-virtual-host logs? I already do that and have no problem with it; I mean separate log files for access, agent, referer.
    – Synetech
    Jan 30, 2013 at 18:24
  • oh - well unless you really have to I wouldn't go there!
    – ETL
    Jan 30, 2013 at 23:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .