Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am running a forward proxy, where I want to forward certain requests to different proxy servers using ProxyRemote. Which request goes to which server depends on many factors (time, load, url, host, user,...) and has to be calculated by an external program.

The process currently works by adding suffix to the host of the request url, which is then matched by ProxyRemoteMatch directives. this is doen by passing the url to a rewrite map.

This works nice, but has the trade of that I have to define EVERY server in the config and DNS.

Here is a working example:

RewriteEngine on
// define an external program that contains the logic where which request should be routed to
RewriteMap balancemap prg:/home/mysite/balancemap.php

<ProxyMatch "http://.*">
        RewriteEngine on        

        // if the route has not yet been set by suffix proxy pattern, pass the request uri and user to the mapping program and store it in an environment variable
        // for example http://google.com will be rewritten to something like http://google.com.5.server.mysite.com - which means it should be routed over server number 5
        RewriteCond %{HTTP_HOST} !\.([0-9]+)\.server\.mysite\.com$ 
        RewriteCond ${balancemap:%{REQUEST_URI}#%{REMOTE_USER}} .*
        RewriteRule .* - [E=BALANCEDURL:%0]

        // if a new url has been calculated, rewrite to it
        RewriteCond %{ENV:BALANCEDURL} .+
        RewriteRule .* %{ENV:BALANCEDURL} [P]
</ProxyMatch>

// now comes the long ist of servers....
// every subdomain is mapped to the correct ip address in internal dns system
ProxyRemoteMatch .*\.1\.server\.mysite\.com.* http://1.server.mysite.com:80
ProxyRemoteMatch .*\.2\.server\.mysite\.com.* http://2.server.mysite.com:80
ProxyRemoteMatch .*\.3\.server\.mysite\.com.* http://3.server.mysite.com:80
ProxyRemoteMatch .*\.4\.server\.mysite\.com.* http://4.server.mysite.com:80
... and so on

Basically what I wish for would be a ProxyRemoteDirective that supports backreferences like this:

ProxyRemoteMatch .*\.(\d+)\.server\.mysite\.com.* http://$1.server.mysite.com:80

Or I could even give the whole IP like this:

ProxyRemoteMatch .*\.(\d+)\.(\d+)\.(\d+)\.(\d+)/.* http://$1.$2.$3.$4:80

For your information: The custom part of the hostname ist stripped off the url again on the forward proxy servers. I would also like to get rid of that, by passing on the url as is!

So if the second argument of the ProxyRemoteMatch directive would interpolate environment variable it woulld look like this:

// now our map just returns the ip of the correct remote server
RewriteCond ${balancemap:%{REQUEST_URI}#%{REMOTE_USER}} .*
RewriteRule .* - [E=REMOVESERVER:%0]
ProxyRemoteMatch .* http://%{ENV:REMOVESERVER}:80

This would be most awesome.

But unfortunately none of the above works.

It took me a while to figure out the current system and I now have about 200 lines of ProxyRemoteMatch directives (200 remote servers). But keeping everything up 2 date with dns and everything is a pain in the ass.

Maby you can also recommend an alternative? Probably the same behaviour can be easily one with other proxy server software? Probably Nginx? Squid?

So any Ideas are welcome!

FYI this is a commercial project, which however I run by myself and it doesn't really make any mentionable profit right now. I plan to make it open source but running my own service with it once its stable.

I think I will also put a bounty on this question ASAP.

share|improve this question
add comment

1 Answer

Given that the following loop (perl, but language 'yyyyy' would do)

foreach my $i (1..200) { 
  print "ProxyRemoteMatch .*\\.$i\\.server\\.mysite\\.com.* http://$1.server.mysite.com:80\n";
}

prints all the long and repetitive config part, why bother? You can maintain the whole shebang with such a one-liner. You may even use Include to wisk it out of the config file altogether. Probably bringin 'make' in the picture would allow you to have it running automathically any time you add/remove a server.

share|improve this answer
    
well yeah, but there are multiple servers with that config and its under version control. also the change requires a webserver restart (reload doesnt suffice) which i would like to avoid... also i think it slows the server down if so many maching directives have to get compiled and evaluated. but generating this by script is what i currently do! one major reason for a dynamic solution would be to get rid of the dns maintenance. unfortunately the remote servers change a lot and a restart every time isnt practicable, so i had to outsource it to dns. –  The Shurrican Jan 5 '12 at 19:36
    
So my second option would be a dynamic page (cgi,ssi, mod_whatever) that does the grunt work. You do not need to restart. As for the slowing down, I'd be surprised if it matters at 200 lines of anything (with reasonably recent HW anyway). Just don't use a database to hold/retriev your proxies :-) –  Alien Life Form Jan 9 '12 at 17:51
    
that sounds promising... i think i am quite capable of figuring out a lot myself once i am on it but could you point me in the right direction how this would look? i am not really familiar with ssi or custom apache modules... –  The Shurrican Jan 11 '12 at 14:59
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.