Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Varnish keeps throwing 503 Service Unavailable when trying to load a page that takes longer to generate on the web server.

In varnishlog I can see an FetchError c http read error: 0 error, though I'm not quite sure what it means.

I also tried increasing the backend timeouts:

backend default {
    .host = "x.x.x.x";
    .port = "80";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}

The backend is an Apache server.

All other pages work fine.

Any ideas?

share|improve this question
add comment

1 Answer

up vote 7 down vote accepted

The error message means (line number references refer to varnish 2.1.3):

While fetching a header [bin/varnishd/cache_fetch.c:399], either:

a) an overflow occurred [bin/varnishd/cache_httpconn.c:170]

or

b) a error occurred when calling read() [bin/varnishd/cache_httpconn.c:175]

The number at the end is the value of errno, so since it's 0 (no error) I would assume that option a) occurred since read() shouldn't return a negative number without setting errno.

The overflow is detected with the following code [bin/varnishd/cache_httpconn.c:167] returning a negative result:

i = (htc->ws->r - htc->rxbuf.e) - 1;    /* space for NUL */

htc->ws is a struct ws [bin/varnishd/cache.h:126] which is a "workspace structure" The r member is the reserved length of that workspace. htc->rxbuf refers to a struct txt [bin/varnishd/cache.h:109] but there's no comment describing what the members (b & e) refer to. Beginning and end perhaps?

I don't know how workspaces are resized (or even if they are) but - and I'm really in guessing territory here - I would assume that some possible causes of the problem are:

  • A very large number of headers
  • Very long headers
  • A bug in how Varnish resizes workspaces (if it does)

It might be useful to try and find the point at which the error can be forced to occur by searching through the space of:

  • Header lengths
  • Numbers of headers

and see if you can reliably reproduce the problem.

You may be able to work around the problem by increasing the http_headers runtime option. (If you're running < 2.1, I think it's a compile-time or configure-time option)

share|improve this answer
    
Now that's an answer! adding -p http_headers=128 at runtime fixed the issue. I can't thank you enough! –  Andrei Serdeliuc Oct 14 '10 at 8:40
    
You're very welcome. I've had similar but different problems so I had done a good chunk of the digging already. :) –  Conor McDermottroe Oct 14 '10 at 8:52
1  
For version 3.0 the header is now 'http_max_hdr' –  kervin Jul 27 '12 at 23:31
1  
I use a Zend Framework application and configured a Wildfire logger to Firebug to track DB queries. On certain occasions it generated a huge amount of queries, and thus headers. This answer helped me find the cause! –  Yeroon Mar 26 '13 at 19:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.