LVS - source IP

Question

I've recently asked a question about my options for load balancing multiple servers (TCP non http traffic, very long running connections with very consistent bandwidth rates): http://serverfault.com/questions/184430/load-balancing-long-running-tcp-connections

In this post I was directed to LVS and just gave that a try. Setup was easy and it pretty much worked right off the bat, with my only issue being that I can only use LVS-NAT (realservers are windows and I do not want to change their config). The problem with that is that it rewrites the source IP, so if a client logs in, I only see the NAT'ed IP and not his original source IP. This poses an issue for us in terms of regulatory requirements as I need to log the real source IP for every client that logs in.

How do others deal with that issue?

Thanks,

Tom

Answer 1

You might reconsider your Windows configuration. I have used direct routing with LVS successfully in Windows. As per the documentation a member of my team wrote:

First install the Windows Loopback Adapter. Start > hdwwiz.exe

Click Next then "Install the hardware that I manually select from a

list (Advanced)

Scroll Down and click "Network Adapters"

Choose Microsoft, then Microsoft Loopback Adapter

Finish the Wizzard

Go to Control Panel\Network and Internet\Network Connections. Rename

the adapters to their descriptive names. Right click on the loopback adapter and manually assign it the LVS VIP.

Go to Start > cmd.exe (right click and choose run as administrator)

Run these Commands.


netsh interface ipv4 set interface "Name of Adapter that holds the real

host IP" weakhostreceive=enabled netsh interface ipv4 set interface "loopback" weakhostreceive=enabled netsh interface ipv4 set interface "loopback" weakhostsend=enabled

This was a Windows 2008 server, which was configured initially using this Web site for guidance.

As far as logging goes, often the only solution will be to utilize the logging at the point in which the client's real IP is still in the route.

With Web traffic, the X_FORWARDED_FOR environment variable could be used. Point being, after a certain point, the network layer cannot be relied on for this information. In that case, you have to move further up the stack for potential solutions.

Answer 2

Yes, Direct Routing is an excellent solution but why did you have a problem with NAT mode? LVS in NAT mode with internal and external subnets is transparent by default i.e. servers can see the clients source IP as if it was a direct connect (just like a firewall). Short description of LVS in NAT mode is here on our site: load balancing methods (NAT mode). BTW you can even do NAT in a one-arm confirguration if you muck around with the Windows routing table.....

Answer 3

When LVS is in NAT mode the LVS box must be a default gateway for the traffic from the real servers precisely because the traffic has the clients real source IP. I don't know how you could get LVS to change the source IP in NAT mode. You should have an LVS box with two network cards (two-arm configuration) , the VIP is on the external subnet and the default gateway for the real servers is on the internal subnet. Traffic is NAT'd transparently by default, i.e. when it turns up at the real server it will look as if it came directly from the clients source IP address. Can you show me a dump of 'ipvsadm -Ln'? Also after you have made a connection to the VIP dumping 'ipvsadm -Lnc' shows you exactly what is going on. Are you sure you are not just seeing 'Health Check' entries in your logs? These would come from the load balancers IP if you are using ldirectord or keepalive for health checks?