Apache losing VirtualHost configurations "randomly"

Question

I am hoping someone might be able to shed some light on this issue that I have had no luck solving elsewhere. We are running an Apache (2.2.15) server used for basic load balancing, but ultimately using ProxyPass directives to our various backend servers.

The problem seems to be that Apache "randomly" no longer has particular virtualhost files properly loaded (or more likely, we are causing this unknowingly), and and we are seeing 404s or proxy errors from Apache.

There are no errors in error_log, I also confirmed that httpd -S still shows the vhost loaded, but the curious part is that if I issue an httpd reload, it starts working again. The server has ~2GB free memory.

We have an Include vhosts/active directive in our httpd.conf file, and within that directory are symlinks to the actual vhost config files (don't beleive that this is related but wanted to disclose), and we are loading about 40 different vhost configs. Here is an example from one in particular that happens relatively requently (it also may be worth mentioning that it seems to only be certain vhost files like this one):

<VirtualHost 10.10.0.77:80>

    ServerName url.mysite.com
    ServerAlias url
    Options +FollowSymLinks
    ProxyPreserveHost On

    ProxyPass / http://hostname.internaldomain.local:8081/
    ProxyPassReverse / http://hostname.internaldomain.local:8081/

</VirtualHost>

<VirtualHost 10.10.0.77:443>
    ServerName url.mysite.com
    ServerAlias url
    RewriteEngine On
    SSLEngine on
    SSLProxyEngine on
    SSLProxyVerify none
    SSLCertificateFile /etc/httpd/conf/ssl/mycert.com.crt
    SSLCertificateKeyFile /etc/httpd/conf/ssl/mycert.key
    SSLCertificateChainFile /etc/httpd/conf/ssl/mycert_chain.crt
    Options +FollowSymLinks
    ProxyPreserveHost On

    ProxyPass / http://hostname.internaldomain.local:8081/
    ProxyPassReverse / http://hostname.internaldomain.local:8081/

</VirtualHost>

from httpd.conf:

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
SetEnv proxy-initial-not-pooled 1

Include vhosts/active/*.active
<Location /balancer-manager>
SetHandler balancer-manager

Order Deny,Allow
Deny from all
Allow from .mydomain.com
</Location>

<IfModule mod_headers.c>
RequestHeader unset Expect early
</IfModule>

Thank you for your help/ideas, as searching on this particular issue has got me nowhere!

Answer 1

I see 2 possible causes:

1. When Apache 2.2 tries to load content behind ProxyPass url and it doesn't respond, then sometimes whole vhost using this ProxyPass stops working. Tested on Debian Squeeze and Wheezy with default Apache 2.2.

2. Your server may be also experiencing random problems with DNS resolver, which affect mapping of ServerName to vhost.

You can try to test, if it's the first cause, by loading 10.10.0.77:80 by IP instead of ServerName.