3

I'm trying to use HAProxy as a fully transparent proxy using TPROXY in Ubuntu 14.04. HAProxy will be setup on the first server with eth0 111.111.250.250 and eth1 10.111.128.134. The single balanced server has eth1 and eth0 as well. eth0 is the public facing network interface while eth1 is for the private network which both servers are in.

Problem: I'm able to connect to the balanced server's port 1234 directly (via eth1) but am not able to reach the balanced server via Haproxy port 1234 (which redirects to 1234 via eth0). Am I missing out something in this configuration?

Removing the line source 0.0.0.0 usesrc clientip from /etc/haproxy/haproxy.cfg, the haproxy works but not transparently.


On the HAProxy server

The current kernel is 3.13.0-24-generic. iptables v1.4.21 is used, which I assume supports TPROXY but am not sure how to check.

The kernel appears to have TPROXY support:

# grep TPROXY /boot/config-3.13.0-24-generic 
CONFIG_NETFILTER_XT_TARGET_TPROXY=m

HAProxy was compiled with TPROXY support:

haproxy -vv
HA-Proxy version 1.5.3 2014/07/25
Copyright 2000-2014 Willy Tarreau <[email protected]>

Build options :
  TARGET  = linux26
  CPU     = x86_64
  CC      = gcc
  CFLAGS  = -g -fno-strict-aliasing
  OPTIONS = USE_LINUX_TPROXY=1 USE_LIBCRYPT=1 USE_STATIC_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built without zlib support (USE_ZLIB not set)
Compression algorithms supported : identity
Built without OpenSSL support (USE_OPENSSL not set)
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

In /etc/haproxy/haproxy.cfg, I've configured a port to have the following options:

listen test1235 :1234
    mode tcp
    option tcplog
    balance leastconn
    source 0.0.0.0 usesrc clientip

    server balanced1 10.111.163.76:1234 check inter 5s rise 2 fall 4 weight 4

The following iptables rules were added

iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 111
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 111 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
sudo ip route flush cache

and

echo 1 >/proc/sys/net/ipv4/ip_forward

On the balanced server

In /etc/networking/interfaces I've set the gateway for eth1 to be the HAProxy box 10.111.128.134 and restarted networking.

auto eth0 eth1
iface eth0 inet static
        address 111.111.250.250
        netmask 255.255.224.0
        gateway 111.131.224.1
        dns-nameservers 8.8.4.4 8.8.8.8 209.244.0.3
iface eth1 inet static
        address 10.111.163.76
        netmask 255.255.0.0
        gateway 10.111.128.134

ip route gives:

default via 111.111.224.1 dev eth0 
10.111.0.0/16 dev eth1  proto kernel  scope link  src 10.111.163.76 
111.111.224.0/19 dev eth0  proto kernel  scope link  src 111.111.250.250 
2
  • I believe I'm having the same problem in serverfault.com/questions/622111/… I believe it's something to do with the routing, not able to find a lot of up-to-date info about it however :/
    – Thermionix
    Aug 21, 2014 at 2:44
  • What's the indication of tcpdump on eth1 on Server B? Are packets arriving? Do they have the correct source IP? Are replies emitted from the interface? If yes, do those appear on eth1 of Server A? Aug 21, 2014 at 23:26

1 Answer 1

1

I believe your issue might be related to not having forwarding or nat rules but you need forward rules that direct traffic between eth0 <-> eth1.

ie:

Chain FORWARD (policy ACCEPT 7601 packets, 661K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  18M   47G ACCEPT     all  --  eth1  eth0    0.0.0.0/0            0.0.0.0/0           
  17M 2922M ACCEPT     all  --  eth0  eth1    0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED

Also one thing that has cough me offguard before was specify the interface device on the mangle table when I'm using multiple devices:

Chain PREROUTING (policy ACCEPT 35M packets, 50G bytes)
 pkts bytes target     prot opt in     out     source               destination         
  59M   52G divert     tcp  --  eth1      *       0.0.0.0/0            0.0.0.0/0            socket

Where it eth1 should not be specify cause we're using eth0/eth1 or even bond0 & eth1.

Lastly you want to make sure you have natting rules ie:

Chain POSTROUTING (policy ACCEPT 1420K packets, 85M bytes)
 pkts bytes target     prot opt in     out     source               destination         
1K  494M PUBLICSNAT     all  --  *      eth1    0.0.0.0/0            0.0.0.0/0           

Chain PUBLICSNAT (1 references)
 pkts bytes target     prot opt in     out   source             destination         
1K  494M SNAT       all  --  *      eth1  10.1.1.0/24         0.0.0.0/0        to:8.8.8.8-8.8.8.10
 1K  336K RETURN     all  --  *      *     0.0.0.0/0            0.0.0.0/0           

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .