0

We're using currently an Apache 2.4 as load-balancer for a multi-master-cluster of three Tomcats-Servers. We can access the internal IP-Address of this load-balancer with an web-browser and change the settings on the fly (e.g. http://xx.xx.xx.xx/balancer-manager). This works and we're pretty fine with this.

Enabled modules:

  • proxy.load (proxy.conf used to configure)
  • proxy_ajp.load
  • proxy_balancer.load (proxy_balancer.conf used to configure)

But now we will host several (three in this setup) virtual hosts on this Apache, each representing a cluster of three Tomcats itself. Each is accessiable via an URL like http://customer.company.tld/app/ui). So the path is the same for each cluster! Now we're facing two problems:

  • The page balancer-manager is only reachable via it's virtual host. Therefore we have to access the balancer-manager via random choosen internal hostname (added to ServerAlias) and add this hostname also into the /etc/hosts off our internal computers to be be able to use it.
  • Furthermore we have to do this for each virtual-host (~cluster). But we want a single page balancer-manager which presents all virtual-hosts and the cluster behind.

Here some sample configuration:
/etc/apache2/site-enabled/foo # we have actually three of them: foo, bar, baz

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName customer-foo.company.tld
    ServerAlias customer-foo-balancer customer-foo.company.tld www.customer-foo.company.tld

    DocumentRoot /var/www

        <Directory />
                Options FollowSymLinks
                AllowOverride None
                Order allow,deny
                deny from all
        </Directory>
        <Directory /var/app/app_static>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        <Directory /var/www/app_static_res>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>

    # load balancer
        <Location /balancer-manager>
                 SetHandler balancer-manager
                 Order deny,allow
                 Deny from all
                 # allows for internal access
                 Allow from 127.0.0.1 ::1 10.1.21.81 10.1.4.9
                 Satisfy all
        </Location>

        ProxyRequests Off
        ProxyVia Off
        ProxyPreserveHost On

        <Proxy balancer://htg>
                ProxySet failonstatus=503
                BalancerMember ajp://10.171.23.120:8010/app lbset=0 route=foo001 loadfactor=40
                BalancerMember ajp://10.171.23.121:8010/app lbset=0 route=foo002 loadfactor=40
                BalancerMember ajp://10.171.23.122:8010/app lbset=0 route=foo003 loadfactor=20
        </Proxy>

        <Proxy balancer://htgservice>
                ProxySet failonstatus=503
                BalancerMember ajp://10.171.23.120:8011/wcs_service route=foo001 loadfactor=40
                BalancerMember ajp://10.171.23.121:8011/wcs_service route=foo002 loadfactor=40
                BalancerMember ajp://10.171.23.122:8011/wcs_service route=foo003 loadfactor=20
        </Proxy>


        ProxyPass        /app balancer://foo stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
        ProxyPassReverse /app balancer://foo

        ProxyPass        /app_service  balancer://fooservice stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
        ProxyPassReverse /app_service  balancer://fooservice

        LogLevel warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

In the past we configured the balancing inside the distinct files proxy.conf and proxy_balancer.conf, which should allow for accessing via the internal IP and show up all clusters and cluster-members on a single page. But this won't work anymore, the configuration for proxies seem to accept only paths (e.g. app and app_service) not URLs or hostnames. We can't and won't change the paths. Therefore we moved the proxy-configuration inside the virtual hosts.

Thanks for your help!

2 Answers 2

3

What about splitting your config. Put your balancer definition outside any virtual host and remove the context:

     <Proxy balancer://htg>
            ProxySet failonstatus=503
            BalancerMember ajp://10.171.23.120:8010 lbset=0 route=foo001 loadfactor=40
            BalancerMember ajp://10.171.23.121:8010 lbset=0 route=foo002 loadfactor=40
            BalancerMember ajp://10.171.23.122:8010 lbset=0 route=foo003 loadfactor=20
    </Proxy>

    <Proxy balancer://htgservice>
            ProxySet failonstatus=503
            BalancerMember ajp://10.171.23.120:8011 route=foo001 loadfactor=40
            BalancerMember ajp://10.171.23.121:8011 route=foo002 loadfactor=40
            BalancerMember ajp://10.171.23.122:8011 route=foo003 loadfactor=20
    </Proxy>

In your vhosts reference these balancers:

ProxyPass        /app balancer://htg/app stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
ProxyPassReverse /app balancer://htg/app

ProxyPass        /app_service  balancer://htgservice/wtg_service stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
ProxyPassReverse /app_service  balancer://htgservice/wtg_service

To access your balancer-manager, create an extra management vhost with <Location /balancer-manager> in it.

This way, you have one place where all your balancers are defined and one vhost to access the balancer-manager which shows you all your balncers.

5
  • Thanks. I will try that as soon as possible! Till now, I thought this kind of split isn't possible and assumend I can work around this with a weird kind of rewrite. Feb 11, 2014 at 13:15
  • I found one drawback. It seems it is not working correctly with stickysessions enabled.
    – Christian
    Feb 12, 2014 at 12:35
  • Perhaps, this is a solution to the sticky session problem: stackoverflow.com/a/9951315/252730
    – Christian
    Feb 12, 2014 at 13:46
  • It is also discribed in this httpd bug report: issues.apache.org/bugzilla/show_bug.cgi?id=39304
    – Christian
    Feb 12, 2014 at 13:55
  • Hi! I already hit this problem yesterday. The styicksessions must be defined with ProxySet inside the <Proxy></Proxy> and it works! Feb 13, 2014 at 8:44
3

Thank you Christian! The solution of Christian is correct!

Basically you have to put ProxyPass and ProxyPassReverse inside the configuration for the Virtual-Hosts, and define the Proxy-Directive in the proxy.conf. As Christian already stated in his own comments, it is required use the ProxySet-Directive inside the Proxy-Directive to configure the StickySessions - it won't work with ProxyPass. Bellow is a short, but complete, example:

Sample VirtualHost (one for foo, bar, baz):

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName customer-foo.company.tld
    ServerAlias customer-foo-balancer customer-foo.company.tld www.customer-foo.company.tld

    DocumentRoot /var/www/foo

    <Directory />
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        deny from all
    </Directory>
    <Directory /var/www/foo/wcs_static>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
     </Directory>
     <Directory /var/www/foo/wcs_static_res>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
     </Directory>

    # maintenance
    RewriteEngine On

    RewriteCond %{DOCUMENT_ROOT}/wcs_static_res/maintenance.html -f
    RewriteCond %{REQUEST_FILENAME} !/wcs_static_res/maintenance.html
    RewriteCond %{REQUEST_FILENAME} !/wcs_static_res/maintenance-time.png
    RewriteCond %{REMOTE_HOST} !^10\.1\.0\.203
    RewriteRule ^.*$    /wcs_static_res/maintenance.html [L]

    # redirect, SSL itself is handled by another server in front of this loadbalancer
    RedirectMatch (.*)/$  https://customer-foo.company.tld/wcs/ui

    # load balancer

    # web-frontend
    ProxyPass        /wcs balancer://foo
    # stickysession can't be set with ProxyPass in this case, see proxy.conf
    ProxyPassReverse /wcs balancer://foo

    # standalone-clients 
    ProxyPass        /wcs_service balancer://fooservice
    ProxyPassReverse /wcs_service balancer://fooservice


    # logging
    LogLevel warn
    ErrorLog ${APACHE_LOG_DIR}/foo_error.log
    CustomLog ${APACHE_LOG_DIR}/foo_access.log combined
</VirtualHost>

proxy.conf (there is only this one):

<IfModule mod_proxy.c>

        ProxyRequests Off
        ProxyVia Off
        ProxyPreserveHost On

        <Proxy balancer://foo>
                # stickysessions must be defined here
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.23.120:8010/wcs lbset=0 route=foo001 loadfactor=40
                BalancerMember ajp://10.171.23.121:8010/wcs lbset=0 route=foo002 loadfactor=40
                BalancerMember ajp://10.171.23.122:8010/wcs lbset=0 route=foo003 loadfactor=20
        </Proxy>
        <Proxy balancer://fooservice>
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.23.120:8011/wcs_service route=foo001 loadfactor=40
                BalancerMember ajp://10.171.23.121:8011/wcs_service route=foo002 loadfactor=40
                BalancerMember ajp://10.171.23.122:8011/wcs_service route=foo003 loadfactor=20
        </Proxy>

        <Proxy balancer://bar>
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.21.110:8010/wcs lbset=0 route=bar001 loadfactor=40
                BalancerMember ajp://10.171.21.111:8010/wcs lbset=0 route=bar002 loadfactor=40
                BalancerMember ajp://10.171.21.112:8010/wcs lbset=0 route=bar003 loadfactor=20
        </Proxy>
        <Proxy balancer://barservice>
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.21.110:8011/wcs_service route=bar001 loadfactor=40
                BalancerMember ajp://10.171.21.111:8011/wcs_service route=bar002 loadfactor=40
                BalancerMember ajp://10.171.21.112:8011/wcs_service route=bar003 loadfactor=20
        </Proxy>

        <Proxy balancer://baz>
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.45.100:8010/wcs lbset=0 route=baz001 loadfactor=40
                BalancerMember ajp://10.171.45.101:8010/wcs lbset=0 route=baz002 loadfactor=40
                BalancerMember ajp://10.171.45.102:8010/wcs lbset=0 route=baz003 loadfactor=20
        </Proxy>

        <Proxy balancer://bazservice>
                ProxySet failonstatus=503 stickysession=JSESSIONID|jsessionid nofailover=Off scolonpathdelim=On
                BalancerMember ajp://10.171.45.100:8011/wcs_service route=baz001 loadfactor=40
                BalancerMember ajp://10.171.45.101:8011/wcs_service route=baz002 loadfactor=40
                BalancerMember ajp://10.171.45.102:8011/wcs_service route=baz003 loadfactor=20
        </Proxy> 
</IfModule>

proxy_balancer.conf:

<IfModule mod_proxy_balancer.c>
        <IfModule mod_status.c>
        <Location /balancer-manager>
                SetHandler balancer-manager
                Order deny,allow
                Deny from all
                # store here your internal IPs for direct access
                Allow from 127.0.0.1 ::1
                Satisfy all
        </Location>

</IfModule>

Now you can change the configuration through http://ipofloadbalancer.com/balancer-manager for all servers behind your proxy. No need to hack with /etc/hosts on the clients and the loadbalancer. Drawback, especially with the Apache 2.4, the interface gets quickly cluttered and you have to scroll much.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .