2

I'm using keepalived to load-balance connections between a number of TCP servers. I don't expect it matters, but the service in this case is rabbitmq. I'm using NAT type balancing with weighted round-robin.

A client connects to the server thus:

[client]-----------[lvs]------------[real server]
            a                b

If a client connects to the LVS and remains idle, sending nothing on the socket, this eventually times out, according to timeouts set using ipvsadm --set. At this point, the connection marked 'a' above correctly disappears from the output of netstat -anp on the client, and from the output of ipvsadm -L -n -c on the lvs box. Connection 'b', however, remains ESTABLISHED according to netstat -anp on the real server box.

Why is this? Can I force lvs to properly reset the connection to the real server?

3 Answers 3

1

Do you have persistent connection enabled? The persistent connection timeout can be set using -p [timeout]

This keeps connection b active to route further requests from the client IP to the same real server.

1
  • 1
    Persistence has nothing to do with timeouts. The persistence control is for how it handles multiple connections from the same ip, and so that it send them all to the same real server.
    – phemmer
    Feb 4, 2011 at 1:36
1

You can use the --set command to decrease the timeout.

/sbin/ipvsadm --set 3600 120 300

Or give haproxy a try.

-1

I had the same problem. I stopped firewalld, and solved it.

in LVS MASTER, I use tcpdump and found that the LVS do not forward the F packet to RS, and client send F package to LVS again and again

add -e to tcpdump to see the MAC of machine:

tcpdump -i eth0 -nn -e host CLIENT_IP and port 80

10.220.16.105 is the client ip, and 10.220.15.10 is VIP

[root@lvs-1 ~]# tcpdump -i eth0 -nn    -e host 10.220.16.105 and port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:54:31.100494 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 74: 10.220.16.105.50290 > 10.220.15.10.80: Flags [S], seq 2199151220, win 29200, options [mss 1460,nop,nop,TS val 3042809285 ecr 0,nop,wscale 7], length 0
20:54:31.100583 fe:fc:fe:ec:9f:61 > fe:fc:fe:e2:15:d6, ethertype IPv4 (0x0800), length 74: 10.220.16.105.50290 > 10.220.15.10.80: Flags [S], seq 2199151220, win 29200, options [mss 1460,nop,nop,TS val 3042809285 ecr 0,nop,wscale 7], length 0
20:54:31.101908 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 3272581466, win 229, options [nop,nop,TS val 3042809285 ecr 101900523], length 0
20:54:31.101940 fe:fc:fe:ec:9f:61 > fe:fc:fe:e2:15:d6, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 3042809285 ecr 101900523], length 0
20:54:31.104153 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 142: 10.220.16.105.50290 > 10.220.15.10.80: Flags [P.], seq 0:76, ack 1, win 229, options [nop,nop,TS val 3042809288 ecr 101900523], length 76: HTTP: GET / HTTP/1.1
20:54:31.104183 fe:fc:fe:ec:9f:61 > fe:fc:fe:e2:15:d6, ethertype IPv4 (0x0800), length 142: 10.220.16.105.50290 > 10.220.15.10.80: Flags [P.], seq 0:76, ack 1, win 229, options [nop,nop,TS val 3042809288 ecr 101900523], length 76: HTTP: GET / HTTP/1.1
20:54:31.104935 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 263, win 237, options [nop,nop,TS val 3042809289 ecr 101900527], length 0
20:54:31.104960 fe:fc:fe:ec:9f:61 > fe:fc:fe:e2:15:d6, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 263, win 237, options [nop,nop,TS val 3042809289 ecr 101900527], length 0
20:54:31.105523 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 280, win 237, options [nop,nop,TS val 3042809290 ecr 101900527], length 0
20:54:31.105547 fe:fc:fe:ec:9f:61 > fe:fc:fe:e2:15:d6, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [.], ack 280, win 237, options [nop,nop,TS val 3042809290 ecr 101900527], length 0
20:54:31.106257 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042809290 ecr 101900527], length 0
20:54:31.306408 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042809491 ecr 101900527], length 0
20:54:31.711239 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042809895 ecr 101900527], length 0
20:54:32.515396 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042810700 ecr 101900527], length 0
20:54:34.130929 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042812312 ecr 101900527], length 0
20:54:37.352335 fe:fc:fe:bf:47:9b > fe:fc:fe:ec:9f:61, ethertype IPv4 (0x0800), length 66: 10.220.16.105.50290 > 10.220.15.10.80: Flags [F.], seq 76, ack 280, win 237, options [nop,nop,TS val 3042815536 ecr 101900527], length 0

finally, i try systemctl stop firewalld.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .