Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have an existing application that we wish to split across multiple servers (for example: 1000 users total, 100 users split across 10 servers).

Ideally, we'd like to be able relay the HTTPS requests to a particular server based on some component of the URL. For example: Users 1 through 100 go to http://server1.domain.com/ Users 2 through 200 go to http://server2.domain.com/ etc. etc. etc.

Where the incoming requests look like this: https://secure.domain.com/user/{integer user # goes here}/path/to/file

Does anyone know of an easy way to do this? Pound looks promising... but it doesn't look like it supports routing based on URL like this.

Even better would be if it didn't need to be hard-coded- The load balancer could make a separate HTTP request to another server to ask "Hey, what server should I relay to for a request to URL {the URL that was requested goes here}?" and relay to the hostname returned in the HTTP response.

share|improve this question

5 Answers 5

Varnish would probably do it. As with other options mentioned here you'd need something like pound in front of it to act as an SSL terminator. However, once done, you can setup each real server as a "backend" and then do add something like the following into the config:

## Define the back end servers.
backend server01 {
    .host = "192.0.2.1";
    .port = "80";
}
backend server02 {
    .host = "192.0.2.2";
    .port = "80";
}

sub vcl_recv {
    if (req.url ~ "^/1[0-9][0-9]/"){
        ## If the first part of the link is 100-199 use server01
        set req.backend = server01;
        pipe;
    } else if (req.url ~ "^/2[0-9][0-9]/") {
        ## If the first part of the link is 200-299 use server02
        set req.backend = server02;
        pipe;
    } else {
        ## If all else fails fall back to server01 
        set req.backend = server01;
        pipe;
    }
}

This is just an extract of the relevant sections and there will probably be more required in the config. For example you could add in the following just after sub vcl_recv { in order to cache any static files so that the servers aren't hit every time for files that don't change.

if (req.request == "GET" && req.url ~ "\.(png|jpg|gif|css)$") {
        lookup;

You can even add in little inline C programs to the config to talk to an external service and decide which backend to use.

share|improve this answer

It would be posible to do this with the URL rewriter in squid - note you'll need to terminate the SSL in front of the proxy (e.g. stunnel).

share|improve this answer
    
Awesome. Any examples of how to do this? It's not a problem to terminate the SSL at the load balancer- all of the server*.domain.com servers are on an internal LAN behind the firewall/load balancer. Off to Google I go! –  Keith Palmer Jan 10 '11 at 16:37

Similar thing could be achieved with nginx. Nginx can do SSL terminations and then proxy your requests to the back-end.

http://www.o3magazine.com/4/a/0/2.html

Is your requirement specific that users 1-100 should go to a particular webserver or can the requests be evenly distributed? I would advise you to evenly distribute them. Would help you spread and thereby lower your risk.

share|improve this answer
    
User accounts are stored on a specific server (it's a legacy app with existing users) so we can't just bounce users to any random back-end server, it has to be to the specific server that houses their account. –  Keith Palmer Jan 10 '11 at 20:56

Do you have coding resources available? I've done almost exactly this using a little program on the middle server. It extracts the user ID from the URI, does a DB lookup to find the location of their files, fetches the file, and then writes it back to the user. I did it using a Java servlet but there are plenty of other options to accomplish the same thing.

share|improve this answer

HAProxy will do what you are asking for. It does have the ability to do Layer 7 routing based on the content of the URL, and it is open source.

share|improve this answer
    
HAProxy does not support SSL (HTTPS) out of the box. SSL support can be added, using fx a patched version of Stunnel, or nginx in front of HAProxy or a number of other ways -- but that's a fair amount of work. –  Jesper Mortensen Jan 12 '11 at 0:45
    
You are correct. HAProxy doesn't support SSL (and patching it to support SSL is generally a bad idea). You should run an SSL terminator in front of it. –  blueben Jan 12 '11 at 0:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.