I've recently asked a question about my options for load balancing multiple servers (TCP non http traffic, very long running connections with very consistent bandwidth rates): Load Balancing long running TCP connections

In this post I was directed to LVS and just gave that a try. Setup was easy and it pretty much worked right off the bat, with my only issue being that I can only use LVS-NAT (realservers are windows and I do not want to change their config). The problem with that is that it rewrites the source IP, so if a client logs in, I only see the NAT'ed IP and not his original source IP. This poses an issue for us in terms of regulatory requirements as I need to log the real source IP for every client that logs in.

How do others deal with that issue?




You might reconsider your Windows configuration. I have used direct routing with LVS successfully in Windows. As per the documentation a member of my team wrote:

First install the Windows Loopback Adapter. Start > hdwwiz.exe

Click Next then "Install the hardware that I manually select from a

list (Advanced)

Scroll Down and click "Network Adapters"

Choose Microsoft, then Microsoft Loopback Adapter

Finish the Wizzard

Go to Control Panel\Network and Internet\Network Connections. Rename

the adapters to their descriptive names. Right click on the loopback adapter and manually assign it the LVS VIP.

Go to Start > cmd.exe (right click and choose run as administrator)

Run these Commands.

netsh interface ipv4 set interface "Name of Adapter that holds the real

host IP" weakhostreceive=enabled netsh interface ipv4 set interface "loopback" weakhostreceive=enabled netsh interface ipv4 set interface "loopback" weakhostsend=enabled

This was a Windows 2008 server, which was configured initially using this Web site for guidance.

As far as logging goes, often the only solution will be to utilize the logging at the point in which the client's real IP is still in the route.

With Web traffic, the X_FORWARDED_FOR environment variable could be used. Point being, after a certain point, the network layer cannot be relied on for this information. In that case, you have to move further up the stack for potential solutions.

  • Thanks for those instructions. I will give it a try in my test environment here. The reason why I'd rather not change anything on the production servers is that they have 4 NIC's that are bonded in pairs of 2 across two switches and 2 vlans and I don't really want to screw up that setup – TJF Sep 28 '10 at 17:56
  • All you're doing is allowing the system to accept the packets for a source IP that isn't otherwise routable on the system. Otherwise, all the routing is normal. I highly encourage you use a test system but it shouldn't be a big deal. – Warner Sep 28 '10 at 17:59
  • Be careful with running Windows Firewall. Depending on how it's configured, if the service isn't running, having the packets dropped will cause ldirectord polling delays, as it causes communication errors. You do not want to filter any ports that you are using with LVS with Windows Firewall. We just discovered this issue recently. Good luck, @Tom Frey. – Warner Oct 1 '10 at 20:39

Yes, Direct Routing is an excellent solution but why did you have a problem with NAT mode? LVS in NAT mode with internal and external subnets is transparent by default i.e. servers can see the clients source IP as if it was a direct connect (just like a firewall). Short description of LVS in NAT mode is here on our site: load balancing methods (NAT mode). BTW you can even do NAT in a one-arm confirguration if you muck around with the Windows routing table.....

  • I do have NAT mode running fine as of now, however, my services do not see the actual source IP and only an NAT'ed source IP. It'd be great if you could elaborate on how I can get LVS in NAT mode to pass on the actual source IP? Thanks! – TJF Jan 6 '11 at 3:34

When LVS is in NAT mode the LVS box must be a default gateway for the traffic from the real servers precisely because the traffic has the clients real source IP. I don't know how you could get LVS to change the source IP in NAT mode. You should have an LVS box with two network cards (two-arm configuration) , the VIP is on the external subnet and the default gateway for the real servers is on the internal subnet. Traffic is NAT'd transparently by default, i.e. when it turns up at the real server it will look as if it came directly from the clients source IP address. Can you show me a dump of 'ipvsadm -Ln'? Also after you have made a connection to the VIP dumping 'ipvsadm -Lnc' shows you exactly what is going on. Are you sure you are not just seeing 'Health Check' entries in your logs? These would come from the load balancers IP if you are using ldirectord or keepalive for health checks?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.