How can I use HAproxy with SSL and get X-Forwarded-For headers AND tell PHP that SSL is in use?

Question

I have the following setup:

(internet) ---> [  pfSense Box  ]    /-> [ Apache / PHP server ]
                [running HAproxy] --+--> [ Apache / PHP server ]
                                    +--> [ Apache / PHP server ]
                                     \-> [ Apache / PHP server ]

For HTTP requests this works great, requests are distributed to my Apache servers just fine. For SSL requests, I had HAproxy distributing the requests using TCP load balancing, and it worked however since HAproxy didn't act as a proxy, it didn't add the X-Forwarded-For HTTP header, and the Apache / PHP servers didn't know the client's real IP address.

So, I added stunnel in front of HAproxy, reading that stunnel could add the X-Forwarded-For HTTP header. However, the package which I could install into pfSense does not add this header... also, this apparently kills my ability to use KeepAlive requests, which I would really like to keep. But the biggest issue which killed that idea was that stunnel converted the HTTPS requests into plain HTTP requests, so PHP didn't know that SSL was enabled and tried to redirect to the SSL site.

How can I use HAproxy to load balance across a number of SSL servers, allowing those servers to both know the client's IP address and know that SSL is in use? And if possible, how can I do it on my pfSense server?

Or should I drop all this and just use nginx?

Answer 1

You dont need to drop it all, you could just use nginx in front of haproxy for SSL support, keeping all your load balancing config. You dont even need to use nginx for HTTP if you don't want to. Nginx can pass both X-Forwarded-For and a custom header indicating SSL is in use (and client cert information if you want). Nginx config snippet that sends required information:

proxy_set_header SCHEME $scheme;      # http/https
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header CLIENT_CERT $ssl_client_raw_cert;

Answer 2

HAProxy can't hit an SSL backend without using raw TCP mode, losing X-Forwarded-For, but, you could potentially re-encrypt the traffic with a listening stunnel for the backend transit. Ugly, though.

I like Ochoto's approach better, with a caveat: nginx is a perfectly capable load balancer; if you're using it, I'd say use it for everything. Proxy your incoming HTTPS to load balanced HTTPS backends - and that way, there's no need for custom headers for SSL information (unless you do need the client certificate).

Answer 3

For anyone else who finds this question, I followed Ochoto's advice and used nginx. Here's the specific steps I used to make this work on pfsense:

1. Using the pfsense web interface, I installed the pfsense PfJailctl package and the "jail_template" package under System > Packages so I could create a FreeBSD jail under which to compile and install nginx on the pfsense system.

2. I configured a jail for my nginx server under Services > Jails, giving the new jail the same hostname and IP address of the virtual IP alias which I had HAproxy running on. O bound the jail to the WAN interface. I used the default jail template and enabled unionfs rather than nullfs.

3. Once the jail had been started, I SSHed in to the pfsense box and ran jls to find the jail's number. I then ran jexec 1 sh to get a shell inside the jail. From there I set up BSD ports.

   portsnap extract
   portsnap fetch update
   cd /usr/ports/www/nginx
   make install clean
   

4. I then configured nginx to listen on port 443, and pass all requests to HAproxy on port 80, including the real IP and the SSL status inside HTTP headers. My usr/local/etc/nginx/nginx.conf looks like:

   worker_processes  1;
   
   events {
       worker_connections  1024;
   }
   
   http {
       upstream haproxy {
           server 209.59.186.35:80;
       }
   
       server {
           listen       443;
           server_name  my.host.name default_server;
           ssl                  on;
           ssl_certificate      my.crt;
           ssl_certificate_key  my.key;
           ssl_session_timeout  5m;
   
           ssl_protocols  SSLv2 SSLv3 TLSv1;
           ssl_ciphers  HIGH:!aNULL:!MD5;
           ssl_prefer_server_ciphers   on;
   
           location / {
               proxy_pass http://haproxy;
   
               proxy_set_header Host $host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   
               proxy_set_header X-Forwarded-Proto https;
           }
       }
   
   }
   

5. I then modified my PHP application to detect the X-Forwarded-Proto HTTP Header:

   function usingSSL()
   {
       return (
          (isset($_SERVER['HTTPS']) && strtolower($_SERVER['HTTPS']) == 'on' )
           || (isset($_SERVER['HTTP_X_FORWARDED_PROTO'])
                  && strtolower($_SERVER['HTTP_X_FORWARDED_PROTO']) == 'https' ));
   }
   

So the final setup is:

(internet) ---> [ -> nginx -> haproxy -]--> (pool of apache servers)
                [    (pfSense server)  ]

Answer 4

You could just use HAproxy in TCP mode and handle SSL with Apache. Less complicated setup that allows HTTP/1.1 throughout.