4

I have successfully set up a load balancing environment using HAProxy and 2 FTP servers running vsftpd. This is what the setup looks so far:

Proxy: ftp00 | 192.168.2.135 (public, eth0) | 10.11.130.1 (private, eth1)
Node01: ftp01 | 10.11.130.140
Node02: ftp02 | 10.11.130.141


Operating system: CentOS 6.6
HA-Proxy: version 1.5.2 2014/07/12
vsftpd: version 2.2.2


ftp00: /etc/haproxy/haproxy.conf

#---------------------------------------------------------------------
# GLOBAL CONFIG
#---------------------------------------------------------------------
global
    daemon
    log         127.0.0.1 local0 info
    log         127.0.0.1 local1 notice
    log         127.0.0.1 local5 debug
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# DEFAULTS CONFIG
#---------------------------------------------------------------------
defaults
        log             global
        mode            tcp
        option          tcplog
        option          dontlognull
        retries         3
        option          redispatch
        option          tcpka
        maxconn         2000
        contimeout      5000

#---------------------------------------------------------------------
# POOL CONFIG
#---------------------------------------------------------------------
listen ftp-lb
        bind 192.168.2.135:21
        mode tcp
        option tcplog
        balance roundrobin
        server ftp01 10.11.130.140:21 weight 10 minconn 30 maxconn 1000 check
        server ftp02 10.11.130.141:21 weight 10 minconn 30 maxconn 1000 check

#---------------------------------------------------------------------
# HAPROXY DASHBOARD CONFIG
#---------------------------------------------------------------------
listen stats
    bind 192.168.2.135:81
    mode http
    stats enable
    stats refresh 30s
    stats show-node
    stats uri  /stats
    stats auth admin:password



ftp00: /etc/sysconfig/iptablesfound [here]

*nat
:PREROUTING ACCEPT [7:724]
:POSTROUTING ACCEPT [5:300]
:OUTPUT ACCEPT [5:300]
-A PREROUTING -d 192.168.2.135/32 -i eth1 -p tcp -m tcp --dport 12001:14000 -j DNAT --to-destination 10.11.130.140
-A PREROUTING -d 192.168.2.135/32 -i eth1 -p tcp -m tcp --dport 16001:18000 -j DNAT --to-destination 10.11.130.141
-A POSTROUTING -s 10.11.130.140/32 -o eth1 -j SNAT --to-source 192.168.2.135
-A POSTROUTING -s 10.11.130.141/32 -o eth1 -j SNAT --to-source 192.168.2.135
COMMIT
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [732:64731]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 81 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT



ftp01: /etc/vsftpd/vsftpd.conf

#------------------------------------------
# GENERAL CONFIG
#------------------------------------------
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
pam_service_name=vsftpd

#------------------------------------------
# LOG CONFIG
#------------------------------------------
xferlog_enable=YES
xferlog_std_format=NO
log_ftp_protocol=YES

#------------------------------------------
# USER WHITELIST
#------------------------------------------
userlist_enable=YES
userlist_deny=NO
userlist_file=/etc/vsftpd/user_list

#------------------------------------------
# PASSIVE MODE CONFIG
#------------------------------------------
#tcp_wrappers=YES
pasv_enable=YES
port_enable=YES
pasv_min_port=12001
pasv_max_port=14000
pasv_address=192.168.2.135
pasv_addr_resolve=NO
connect_from_port_20=YES

#------------------------------------------
# Added listen address for internal only
#------------------------------------------
listen=YES
listen_address=10.11.130.140

#-----------------------------------------
# BANNER CONFIG
#-----------------------------------------
banner_file=/etc/vsftpd/issue



ftp01: /etc/sysconfig/iptables

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 12001:14000 -j ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 21 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 20 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT



ftp02: /etc/vsftpd/vsftpd.conf

Difference to ftp01 config:

pasv_min_port=16001
pasv_max_port=18000

listen_address=10.11.130.141



ftp02: /etc/sysconfig/iptables

Same as **ftp01** with respective port ranges



all nodes: /etc/sysconfig/iptables-config

IPTABLES_MODULES="nf_conntrack_ftp"



SELinux is deactivated on all machines. I've followed several tutorials (like this, and this), but I still can't get passive mode to work. I can log in to the FTP servers via the HAproxy load balancer (which is set to roundrobin and this also works), and it keeps giving me this:

220-***FTP SERVER CLUSTER NODE 02***
220
Name (192.168.2.135:root): root
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
227 Entering Passive Mode (192,168,2,135,67,94).
ftp: connect: Connection timed out
ftp>


Since I've gone through all I can think of and almost everything that's out there dealing with this, I am getting a little frustrated. My configuration is built identically to those in the tutorials, but it just won't work. Maybe I am missing out on something that I didn't notice yet, so any help is much appreciated!

At least the port number seems to be correct. ftp02 is set up to use 16001 - 18000, and entering passive mode uses 67*256 + 94 = 17246, which is totally fine.

4
  • I thought about that the issue could be I am only using 1 single eth interface. Can anyone confirm/disprove that this might be the cause? Jan 30, 2015 at 23:52
  • Added another NIC, same issue. Feb 2, 2015 at 17:22
  • Checked the tcpdumps on all 3 machines: logging in on LB shows tcpdump send/response to the client IP. Doing ls after logging in doesn't show any kind of tcpdump on the respective FTP node I am connected to. I still can't seem to figure out what's the issue here. Feb 2, 2015 at 22:35
  • I also tried adding the port range for the backend servers to the haproxy.cnf as follows, but with no success: listen ftp-lb bind 192.168.2.135:21,:6900-7099 Feb 3, 2015 at 16:26

2 Answers 2

1

Based on RH docs I've read relating to Passive FTP LB, you may need to enable a kernel module

# modprobe ip_vs_ftp 

Red_Hat_Enterprise_Linux-6-Load_Balancer_Administration-en-US

In order to enable passive FTP connections, ensure that you have the ip_vs_ftp kernel module loaded, which you can do by running the command modprobe ip_vs_ftp as an administrative user at a shell prompt.

I haven't confirmed this but will be diving into this type of HA ftp service very soon, so hope your write up plus this loaded module is correct!

0

This should be fairly simple:

a) You must configure the passive mode ip as x.135 (which you have done)

b) You must restrict the port range on the ftp server (which you have done)

c) You must bind haproxy to ALL relevant ports (your config doesn't show that) Be careful to remove the port from each real server when you do this i.e.

10.11.130.140:21

becomes

10.11.130.140

You almost certainly need to enable persistence/sticky in the haproxy config... and longer timeouts - 15mins?

But more importantly when testing - Turn off the firewall stuff completely test if your haproxy config works from the local network - once you are happy that is all working. Then start configuring your firewall. Do you really need a local stateful inspection on each FTP server? Just configure your external firewall correctly.

Hope you get it working.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .