ELB reports backend_status_code of "0" for some requests

Question

I found some weird entries in my ELB logs that I can't really explain. They all look like this (parsed into JSON):

{
    "backend": "-",
    "backend_processing_time": "-1",
    "backend_status_code": 0,
    "client_ip": "XXX.XXX.XXX.XXX",
    "client_port": "52084",
    "elb": "ELB1-PROD",
    "elb_status_code": 504,
    "http_method": "POST",
    "http_version": "HTTP/1.1",
    "received_bytes": 16051,
    "request_processing_time": "-1",
    "request_uri": "https://example.com:443/example/",
    "response_processing_time": "-1",
    "sent_bytes": 0,
    "ssl_cipher": "ECDHE-RSA-AES128-GCM-SHA256",
    "ssl_protocol": "TLSv1.2",
    "timestamp": "2015-10-29T04:41:47.251500Z",
    "user_agent": "client/4.0"
}

e.g. "backend_processing_time": "-1" or "backend_status_code": 0 seem really weird.

Any idea what this could mean? I suspected that the backend servers were overloaded, but we didn't experience a higher load than usual when these errors happened, and the errors have all been triggered by the same client IP.

Answer 1

When an ELB returns a 504 Gateway Timeout it indicates that the load balancer closed a connection because a request did not complete within the idle timeout period (according to the docs).

So, the ELB did not receive a response within the idle timeout limit resulting in the connection being killed and a 504 response being returned to the client. With the knowledge that the backend request never finished in mind, it actually makes sense that the backend_status_code is 0 and that the backend_processing_time is -1. The ELB can't answer something it doesn't know!