Nginx log request and response API Proxy

Question

I have a configuration to redirect all my local API calls (from a client, with a apikey header) at example.com to a remote API server distant-api-server.com:8000).

upstream api-server {
    server distant-api-server.com:8000;
}

server {
    listen 80;
    index index.html;
    server_name example.com;

    location ~ /api/(?<path>.*) {                                               

    if ($request_method = OPTIONS ) {                                       
        add_header 'Access-Control-Allow-Origin' "$http_origin";            
        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';     
        add_header 'Access-Control-Allow-Headers' 'apikey';                 
        return 200;                                                         
    }                                                                       

    if ($api_route = "error"){return 501;}                                  

        error_page 501 /501_apikey.html;                                    

        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, PUT';
        add_header 'Access-Control-Allow-Headers' 'apikey';
        proxy_set_header X-Real-IP $remote_addr;                            
        proxy_set_header X-NginX-Proxy true;                                
        proxy_pass http://$api_route/$path$is_args$args;                    
        proxy_set_header X-Forwarded-For $remote_addr;                      
        proxy_set_header X-Forwarded-Host $remote_addr;                     
        proxy_ssl_session_reuse off;                                        
        proxy_set_header Host $http_host;                                   
        proxy_redirect off;                                                 
    }
}

I'd like to log: - Request send from local server to /api - Request as sent then by Nginx to http://$api_route/$path$is_args$args - Response received from distant API and transmitted by Nginx to my client page

For the moment when I make calls to /api, an error on the distant API and then my request time out. By knowing what is sent/received by Nginx would help me debug.

How can I achieve that kind of logging?

Answer 1

OP, Welcome to Server Fault! As an FYI, listen to @MichaelHampton’s advice and triple check your instincts. In this case the access log really does provide you with the information you likely want. You just have to tell it what it is that you want logged.

To expand on M Hampton’s suggestion, what you should do is define your own log format for upstream/proxy logging and then add the access_log directive to the location from which you are doing proxy_pass using your custom log format.

location ~ /api/(?<path>.*) {
    log_format upstream_logging . . .;
    . . .
    access_log /var/log/nginx/api_logging.log upstream_logging;
}

And you can include whatever variables from nginx you want to in your log format. You will have to specify which headers you want individually from the upstream using the variables here, for example $upstream_http_server. Here is one proposed logging format for logging upstream information that you can use as a starting point.

log_format upstream_logging '[$time_local] $remote_addr - $remote_user - $server_name to: $upstream_addr: $request upstream_response_time $upstream_response_time msec $msec request_time $request_time';

A little bit of googling and reading the docs (and listening to M Hampton!) will take you a long way. Good luck!