1

I have a custom header "AB.CD". I want to log this header value in my nginx access logs.

This is the log format that I want to try in nginx.conf :

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" "$http_AB.CD" '

However the dot (period) seems to be unacceptable. I tried escaping it also, but of no use. It logs the data as ... "-" "-.CD"

What is the right way of logging a header that has a dot in it?

  • 1
    I'm pretty sure a "dot" is a valid character in header names, but none of the standard values have them. I think your life will be easier if you use a different character. – Tim Jun 16 '17 at 4:07
  • I'm not sure, but try ${http_ab.cd} – Alexey Ten Jun 16 '17 at 4:57
  • @Tim I agree. But until I get to change it. – TJ- Jun 16 '17 at 5:16
  • @AlexeyTen: Thanks, but it didn't work. – TJ- Jun 16 '17 at 5:16
2

Although the period is indeed a valid character for an HTTP header, it appears that nginx is not able to handle it correctly. This goes beyond simply logging the value to a file.

Before trying to log this custom header, make sure the header is actually available to be logged! In this case, it appears that nginx does not recognize this as being a valid header.

Try setting a custom header and running this simple PHP script with the following (example from php.net):

<?php
foreach (getallheaders() as $name => $value) {
    echo "$name: $value<br/>\n";
}

This will display a readable list of all the headers in the request.

Now, using these web developer tools, I attempted to set a custom HTTP header with a period:

Each of these tools behaved the same way: HTTP headers with normal names (like AB-CD) worked as expected; HTTP headers with names like AB.CD or AB%CD were not recognized by nginx, and were not shown in the output of the above script.

The above applies to nginx-1.10.3, nginx-1.11.8, nginx-1.12.0, and nginx-1.13.1.

  • Appreciate your effort investigating the thing. However, this isn't an answer to the query - and yes, of course, the header is available to be logged! – TJ- Jun 20 '17 at 5:06
  • OK. So, the header is available to be logged. Does that mean it shows up in the output of getallheaders()? How does the header get set in your environment? Is it done by a third-party app, a script, or some plugin? – JonathanDavidArndt Jun 30 '17 at 3:14
  • Yes, it does. How does the source matter at all? No - It is not set by any 3rd party app. – TJ- Jun 30 '17 at 4:16
  • The source does not matter for your end result (logging). But this is an interesting problem, and I am currently unable to investigate further, because I am unable to even set headers like this from any source. It would help me further diagnose the problem. – JonathanDavidArndt Jun 30 '17 at 11:13
  • Use a Rest Client chrome plugin. You can send any header you'd want to. – TJ- Jul 1 '17 at 7:10
0

Try using the escape parameter in your log format:

From the [nginx documentation]:1

The escape parameter (1.11.8) allows setting json or default
characters escaping in variables, by default, default escaping
is used.
  • It didn't work. i.e. I was able to translate my logs to a json format, but it continues to use the period as a concatenation operator and does not escape it. – TJ- Jun 16 '17 at 5:56
0

Documentation is all well and good, but the ultimate source of truth is the source code, and potentially it can even be easier to find what you are after that way. Or not - YMMV.

The ngx_http_log_compile_format() function in http://lxr.nginx.org/source/src/http/modules/ngx_http_log_module.c is the bit that parses your log_format directive.

1603               if ((ch >= 'A' && ch <= 'Z')
1604                         || (ch >= 'a' && ch <= 'z')
1605                         || (ch >= '0' && ch <= '9')
1606                         || ch == '_')
1607                     {
1608                         continue;
1609                     }
1610 
1611                     break;

So that bit is clear enough - the variable name in your log_format directive can only contain alphanumeric plus '_'.

That's not the end of story though, because there's still the question of how a header containing a '.' is mapped into a variable name.

http://lxr.nginx.org/source/src/http/ngx_http_parse.c does the parsing in ngx_http_parse_header_line(). There's a bit more to read than I really want to go into though. It appears that if the code hits a '.' in the field name, nothing gets done to the hash of the field name. r->invalid_header = 1; gets set, but it doesn't return NGX_HTTP_PARSE_INVALID_HEADER; as in some other cases. (see code lines 922-982).

I don't care to go on to exploring via the code as to what r->invalid_header = 1; does, and it doesn't seem necessary. log_format ... ${http_abcd} ... might work, and if that doesn't work, then I doubt anything will. I'd just try it, and if that doesn't work, assume nothing will.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.