Outbound host cluster for File Transfers (SFTP and FTP)

Question

I am trying to set up a cluster of machines that handle outbound file transfers using SFTP and FTP. Given fixed IP whitelisting requirements on the other end of these connections I need all the servers in my cluster to look like the same machine (host/ip address). Anyone know the best way to set this up? Machines are running Windows Server 2008 R2 and Windows Server 2012 R2.

For inbound connections into my environment I have a working solution using a network load balancer (Brocade Virtual Traffic Manager), but for outbound connections however, I am stuck because my load balancer only handles inbound connections. While I am using third party software to originate the SFTP and FTP requests, its essentially SFTP and FTP connections that I am trying to make look like one host/IP to the receiving servers and if I can handle it at the service layer I should be able to get my third party applications to leverage the solution.

This is an enterprise file transfer operations system that has been in place for over a decade so reaching out to all our partners and clients to have them update their firewalls for additional/changed IP address for inbound connections is really a non-starter.

Anyone have any ideas as to how to get this to work? Cant be the first time someone has run into this requirement I would think. Thank you in advance.

Answer 1

Depending on which tools you use to do the transfer work with SFTP or FTP, you might want to look into the topic of SOCKS proxying.

The SOCKS protocol was designed to facilitate generic proxying and strong authentication, for most kinds of network connections. Some refer to it as Layer 4 proxying, since the application may know about and actively support using such facilities.

There's quite a lot of documentation on SOCKS in general, but some of it refers to the older version 4. Check RFC 1928 for the specification of the version 5 SOCKS protocol, which is the most used (based on hearsay, mind you). It is quite common for network tools to make use of SOCKS5, and OpenSSH is a popular example.

Running a SOCKS5 proxy on your internal network, and using the IP address your partners and clients know for outgoing connections should do the trick. Most SFTP and FTP tools I've used support adding a proxy somewhere in the configuration. 'Generic proxy' seems to be what people call it there.

A popular proxy server with a clean protocol implementation is 'Dante', made by Inferno Nettverk of Norway ( https://www.inet.no/dante/index.html ) and released under a BSD license. Caveat: only available for Linux/Unix.