0

I am hoping someone might be able to shed some light on this issue that I have had no luck solving elsewhere. We are running an Apache (2.2.15) server used for basic load balancing, but ultimately using ProxyPass directives to our various backend servers.

The problem seems to be that Apache "randomly" no longer has particular virtualhost files properly loaded (or more likely, we are causing this unknowingly), and and we are seeing 404s or proxy errors from Apache.

There are no errors in error_log, I also confirmed that httpd -S still shows the vhost loaded, but the curious part is that if I issue an httpd reload, it starts working again. The server has ~2GB free memory.

We have an Include vhosts/active directive in our httpd.conf file, and within that directory are symlinks to the actual vhost config files (don't beleive that this is related but wanted to disclose), and we are loading about 40 different vhost configs. Here is an example from one in particular that happens relatively requently (it also may be worth mentioning that it seems to only be certain vhost files like this one):

<VirtualHost 10.10.0.77:80>

    ServerName url.mysite.com
    ServerAlias url
    Options +FollowSymLinks
    ProxyPreserveHost On

    ProxyPass / http://hostname.internaldomain.local:8081/
    ProxyPassReverse / http://hostname.internaldomain.local:8081/

</VirtualHost>

<VirtualHost 10.10.0.77:443>
    ServerName url.mysite.com
    ServerAlias url
    RewriteEngine On
    SSLEngine on
    SSLProxyEngine on
    SSLProxyVerify none
    SSLCertificateFile /etc/httpd/conf/ssl/mycert.com.crt
    SSLCertificateKeyFile /etc/httpd/conf/ssl/mycert.key
    SSLCertificateChainFile /etc/httpd/conf/ssl/mycert_chain.crt
    Options +FollowSymLinks
    ProxyPreserveHost On

    ProxyPass / http://hostname.internaldomain.local:8081/
    ProxyPassReverse / http://hostname.internaldomain.local:8081/

</VirtualHost>

from httpd.conf:

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
SetEnv proxy-initial-not-pooled 1

Include vhosts/active/*.active
<Location /balancer-manager>
SetHandler balancer-manager

Order Deny,Allow
Deny from all
Allow from .mydomain.com
</Location>

<IfModule mod_headers.c>
RequestHeader unset Expect early
</IfModule>

Thank you for your help/ideas, as searching on this particular issue has got me nowhere!

1

I see 2 possible causes:

  1. When Apache 2.2 tries to load content behind ProxyPass url and it doesn't respond, then sometimes whole vhost using this ProxyPass stops working. Tested on Debian Squeeze and Wheezy with default Apache 2.2.

  2. Your server may be also experiencing random problems with DNS resolver, which affect mapping of ServerName to vhost.

You can try to test, if it's the first cause, by loading 10.10.0.77:80 by IP instead of ServerName.

  • Thanks for the ideas Tomasz. Curious as to whether either one of these scenarios would log to error_log? I've got Apache set to debug, but it looks like you can only do it on a module basis in 2.4+. I am going to test those scenarios and see if that sheds some light. Can you elaborate on the 2nd point? my issue w/ that is I need to use ServerName bc I am doing name-based vhosting, all on the same IP. I'm confident external DNS is good, but to your point, internal DNS could have issue for the ProxyPass host (but doesn't appear to be the case when testing pings). Other logs to catch this? – Chase Jun 17 '15 at 20:58
  • 1
    In my case, neither of these problems were properly logged in error_log, however the first was reported instead of target web page first time it happened after Apache start (only once, next times blank page was returned). As for DNS, I didn't mean configuration, but stability. I spotted this because "apt-get update" also had occasional problems, on Apache side there were just random 5xx error codes in that moments. – Tomasz Klim Jun 18 '15 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.