Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've got an Apache 2.0.52 server on CentOS 4 that front-ends a couple of App servers (mix of Jetty and Tomcat). Apache has a handful of virtual hosts configured like this:

<VirtualHost www1.example.com:443>
    ServerName www1.example.com
    DocumentRoot "/mnt/app_web/html"

    SSLEngine on
    SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP
    SSLCertificateFile      /etc/httpd/conf/ssl.crt/server.crt
    SSLCertificateChainFile /etc/httpd/conf/ssl.crt/chain.crt
    SSLCertificateKeyFile   /etc/httpd/conf/ssl.key/server.key
    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0

    RewriteEngine on
    RewriteRule ^/app1/(.*)$ http://app1.example.com:8080/app1/$1 [P,L]
    RewriteRule ^/app2/(.*)$ http://app2.example.com:8080/app2/$1 [P,L]
</VirtualHost>

However, I'm getting the following errors in the logs intermittently:

[Fri Dec 04 07:19:41 2009] [error] (113)No route to host: proxy: HTTP: attempt to connect to 10.0.0.1:8080 (app1.example.com) failed

I initially tried turning off IPv6, and that seemed to largely cure it, but I still have sporadic bursts of these messages.

Additionally, we're running memcache on same front-end and during the times when I'm getting those messages in Apache's log, the following command doesn't work:

echo stats | nc 127.0.0.1 11211

No messages are printed, but neither are the stats printed. I am completely lost as to how to proceed with troubleshooting this. =(

share|improve this question

1 Answer 1

up vote 0 down vote accepted

To solve this problem you need to add rule(s) in the 'iptables' of your App servers. For Red Hat Enterprise the file is '/etc/sysconfig/iptables' . It should be the same for CentOS.

You probaly have one or more rules that accept NEW connection from the front-ends that look like these:

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp -s 'IP of the front-ends' --dport 'port number' -j ACCEPT

OR

-A RH-Firewall-1-INPUT -m state --state NEW -m multiport -m tcp -p tcp -s 'IP of the front-ends' --dports 'ports numbers' -j ACCEPT

Your problem shoud be solved by adding rules that send a tcp-reset to the front-ends for each SYN packet that passed throught the preceedings rules. The rules should looks like these:

-A RH-Firewall-1-INPUT -m tcp -p tcp -s 'IP of the front-ends' --dport 'port number' --syn -j REJECT --reject-with tcp-reset

OR

-A RH-Firewall-1-INPUT -m multiport -m tcp -p tcp -s 'IP of the front-ends' --dports 'ports numbers' --syn -j REJECT --reject-with tcp-reset

Add the rules near the end of your 'iptables' just before the rule that looks like:

-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited

Good luck.

Paul

share|improve this answer
    
While this didn't exactly fix the problem, it did point the way to a proper resolution - specifically the iptables rules allowed access to tomcat only if the request was IPv4 (d'oh!) We also shuffled a couple of PHP apps off the front-end server to another VM and we've been error-free for a couple of months now. Thanks! (I'd upboat but I don't have the rep on this shard =/) –  BonkaBonka Sep 9 '10 at 17:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.