Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

When hosting multiple domains with apache it's useful to see the logwatch apache output with the virtual host name included, but I only get:

 --------------------- httpd Begin ------------------------

 Requests with error response codes
   400 Bad Request
      /: 1 Time(s)
      /robots.txt: 1 Time(s)

whereas I would like something like

 --------------------- httpd Begin ------------------------

 Requests with error response codes
   400 Bad Request
      example.com/: 1 Time(s)
      example.org/robots.txt: 1 Time(s)

How can I achieve this with logwatch?

share|improve this question
    
I have this same issue. Did you ever resolve this? –  dunxd Jan 3 '11 at 23:49

6 Answers 6

up vote 2 down vote accepted

Try this (works for me): Define LogFormat in your httpd.conf as

LogFormat "%h %t [%V] \"%r\" %>s \"%{Referer}i\""

With this particular case, you'll have remote_address, date/time, [The server name according to the UseCanonicalName setting], request, satus code and Referer (that's my desired format) and then put

$LogFormat "%h %t %V \"%r\" %>s \"%{Referer}i\""

in your services/http.conf LogWatch file. That will

  1. make apache put the hostname (Canonical or not, it depends if you use %v or %V)
  2. force LogWatch to understand your Apache Acces log

Here is an example of the a line in the log output with this particular set of directives:

172.3.20.11 [01/Jun/2011:21:00:52 +0200] joomla.local "GET /images/tabs_back.png HTTP/1.1" 404 "http://joomla.local/templates/beez_20/css/personal.css"

If we FOCUS ON ERROR CODES, and how are they treated in LogWatch, here are some changes you can made to /usr/share/logwatch/scripts/services/http: Add:

my $my_host = ""; my $my_url = "";

Then, about line 462, add this line to save our 4th column (HOST):

$field{my_host} = $field{$log_fields[3]};

And in line 560, after fmt_url is shorten ( if (length($field{url}) > 60) {...} ) add:

$my_host = $field{$log_fields[3]}; `
$my_host = substr($my_host,1);`
$my_url=$my_host . $fmt_url;`

Finally, change:

$needs_exam{$field{http_rc}}{$fmt_url}++;

by

$needs_exam{$field{http_rc}}{$my_url}++;

doing so, you'll have this in your Logwatch:

Requests with error response codes
404 Not Found
joomla.local/images/tabs_back.png: 3 Time(s)

I Hope it helps all you out

share|improve this answer
    
Is the difference between the two definitions "[%V]" vs "%V" intended? Trying to get this to work but no luck so far. I notice the domain in the example log output doesn't have square brackets. –  contrebis May 12 '12 at 12:23

Many thanks to @Syquus who put me on the right path to modifying the /usr/share/logwatch/scripts/services/http file.

My file and solution was different but I thought I would share all the same.

I use the standard vhost_combined LogFormat that Apache provides that looks like:

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined

which outputs something like:

example.org:80 1.1.1.1 - - [08/Oct/2013:16:55:01 +0000] "GET / HTTP/1.1" 200 6094 "-" "Opera/9.80 (X11; Linux x86_64; Edition Linux Mint) Presto/2.12.388 Version/12.16"

I put this in the service configuration override at /etc/logwatch/conf/services/http.conf:

$logformat = "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\""

After finding the approximately correct places to make the changes for @Syquus's solution in /usr/share/logwatch/scripts/services/http, I thought simply changing the index from [3] to [0] would work - it didn't. I got incorrect segments of the path and even after traversing the whole hash/array, I didn't find the hostname. Debugging was frustrating because I'm new to Perl, but my solution was to add in matching for the %v which was being discarded and then modifying the url further down to include the domain name.

Diff for my solution (I also removed the url truncation), YMMV:

--- __http.2013-10-09   2013-10-09 13:11:48.000000000 +0000
+++ http        2013-10-09 14:36:59.000000000 +0000
@@ -132,6 +132,8 @@
 #  Build tables of the log format to parse it and determine whats what
 #

+my $my_url = "";
+
 my $detail = $ENV{'LOGWATCH_DETAIL_LEVEL'} || 0;
 my $ignoreURLs = $ENV{'http_ignore_urls'};
 my $ignoreIPs = $ENV{'http_ignore_ips'};
@@ -379,7 +381,10 @@
 $logformat =~ s/%[\d,!]*/%/g;
 while ($end_loop) {

-   if ($logformat =~ /\G%h/gc) {
+   if ($logformat =~ /\G%v/gc) {
+      $parse_string[$parse_index] .= "(\\S*?)";
+      $parse_field[$parse_index][$parse_subindex++] = "my_host";
+   } elsif ($logformat =~ /\G%h/gc) {
       $parse_string[$parse_index] .= "(\\S*?)";
       $parse_field[$parse_index][$parse_subindex++] = "client_ip";
    } elsif ($logformat =~ /\G%l/gc) {
@@ -437,7 +442,6 @@
 #
 #  Process log file on stdin
 #
-
 while (my $line = <STDIN>) {
    chomp($line);

@@ -580,11 +584,12 @@
          !((defined $ignoreURLs) && ($field{url} =~ /$ignoreURLs/)) &&
          !((defined $ignoreIPs) && ($field{client_ip} =~ /$ignoreIPs/)) ) {
       my $fmt_url = $field{url};
-      if (length($field{url}) > 60) {
-         $fmt_url = substr($field{url},0,42) . " ... " .
-                    substr($field{url},-15,15);
-      }
-      $needs_exam{$field{http_rc}}{$fmt_url}++;
+      #if (length($field{url}) > 60) {
+      #   $fmt_url = substr($field{url},0,42) . " ... " .
+      #              substr($field{url},-15,15);
+      #}
+      $my_url = $field{my_host} . $fmt_url;
+      $needs_exam{$field{http_rc}}{$my_url}++;
    }
    if (defined $field{userid} && $field{userid} ne "-" &&
          (eval $user_display) &&

Should I decide to serve up secured content or content on a port other than :80 I might include it in the future. It should be obvious now how.

Hope this helps!

UPDATE

Made some more changes, fixed a bug. Rather than keep editing this answer you can find my modifications here: https://bitbucket.org/ubiquitypress/logwatch

share|improve this answer

I found a blog post detailing this problem and how someone resolved it. Don't know how this will affect log analysis, but noting it here as I found it useful.

share|improve this answer

You can get the hostname using the LogFormat directive in the apache configuration. You can use the following option

%{Host}i

I found this information in this link. Logwatch should be able to parse custom information.

share|improve this answer

I don't think it is possible if you are logging all virtual domains into the same log file... The apache log will not differentiate between them.

I would also suggest that you take a look at the open source OSSEC. We moved from logwatch to it, because it is in real time and allow centralized correlation (correlating things like ssh failed login with apache 400 errors).

share|improve this answer

Ignor 1st step

i thing u can use this http://httpd.apache.org/docs/1.3/logs.html

share|improve this answer
1  
I should downvote this as well but I don't find it fair to do that you twice in one question. Please try to understand the problem before throwing some generic links which are the first results in your google search. –  Robert Munteanu Jun 12 '09 at 9:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.