2

I have an internet connection issue with several computers on a KVM serer, which is also having the same issue. The internet connection on Windows guests works fine. This is happening on multiple machines. The gateway is a pfsense firewall with all LAN side traffic allowed as per its default configuration, the 192.168.1.1 router uses its DMZ to push all traffic to the firewall.

I've tested connectivity with ping.

$ ping 216.58.220.100
PING 216.58.220.100 (216.58.220.100) 56(84) bytes of data.
64 bytes from 216.58.220.100: icmp_seq=1 ttl=56 time=49.3 ms
64 bytes from 216.58.220.100: icmp_seq=2 ttl=56 time=51.5 ms
64 bytes from 216.58.220.100: icmp_seq=3 ttl=56 time=49.8 ms
64 bytes from 216.58.220.100: icmp_seq=4 ttl=56 time=84.2 ms
64 bytes from 216.58.220.100: icmp_seq=5 ttl=56 time=49.1 ms
^C
--- 216.58.220.100 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 49.139/56.820/84.288/13.762 ms

Nslookup works fine

$ nslookup google.com
Server:         10.0.100.1
Address:        10.0.100.1#53

Non-authoritative answer:
Name:   google.com
Address: 216.58.220.110

I can telnet on port 80

$ telnet google.com 80
Trying 216.58.220.110...
Connected to google.com.
Escape character is '^]'.

However if I try to send a command, such as GET / HTTP/1.0 to see if it would return the HTTP header, nothing happens.

I ran a packet capture on the firewall against the sevrers IP as I was trying the HTTP GET command and cam eup with this, but I'm not too sure what to make of it.

1   0.000000    IntelCor_93:8f:4b   Broadcast   ARP 60  Who has IP.IP.IP.10?  Tell IP.IP.IP.200
2   2.130120    IP.IP.IP.10 216.58.220.100  TCP 66  55642?80 [FIN, ACK] Seq=1 Ack=1 Win=229 Len=0 TSval=19535544 TSecr=1347772624
3   3.816772    IP.IP.IP.10 IP.IP.IP.1  DNS 74  Standard query 0x13ec  A www.google.com
4   3.816839    IP.IP.IP.10 IP.IP.IP.1  DNS 74  Standard query 0xf326  AAAA www.google.com
5   4.003369    IP.IP.IP.1  IP.IP.IP.10 DNS 102 Standard query response 0xf326  AAAA 2404:6800:4006:801::2004
6   4.035623    IP.IP.IP.1  IP.IP.IP.10 DNS 90  Standard query response 0x13ec  A 216.58.220.100
7   4.036683    IP.IP.IP.10 216.58.220.100  TCP 74  55643?80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=19536021 TSecr=0 WS=128
8   4.086164    216.58.220.100  IP.IP.IP.10 TCP 74  80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347876636 TSecr=19536021 WS=128
9   4.086474    IP.IP.IP.10 216.58.220.100  TCP 66  55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536033 TSecr=1347876636
10  4.548559    216.58.220.100  IP.IP.IP.10 TCP 74  [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347877098 TSecr=19536021 WS=128
11  4.548808    IP.IP.IP.10 216.58.220.100  TCP 66  [TCP Dup ACK 9#1] 55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536149 TSecr=1347876636
12  6.547488    216.58.220.100  IP.IP.IP.10 TCP 74  [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347879098 TSecr=19536021 WS=128
13  6.547774    IP.IP.IP.10 216.58.220.100  TCP 66  [TCP Dup ACK 9#2] 55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536649 TSecr=1347876636
14  8.532081    IP.IP.IP.10 216.58.220.100  TCP 82  [TCP segment of a reassembled PDU]
15  8.781690    IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537208 TSecr=1347876636
16  9.033636    IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537271 TSecr=1347876636
17  9.453382    IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] [TCP segment of a reassembled PDU]
18  9.537284    IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537397 TSecr=1347876636
19  10.544897   IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537649 TSecr=1347876636
20  10.545286   216.58.220.100  IP.IP.IP.10 TCP 74  [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347883098 TSecr=19536021 WS=128
21  10.545542   IP.IP.IP.10 216.58.220.100  TCP 66  [TCP Dup ACK 19#1] 55643?80 [ACK] Seq=17 Ack=1 Win=29312 Len=0 TSval=19537649 TSecr=1347876636
22  12.563760   IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19538154 TSecr=1347876636
23  16.601741   IP.IP.IP.10 216.58.220.100  TCP 82  [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19539164 TSecr=1347876636
24  18.540457   216.58.220.100  IP.IP.IP.10 TCP 74  [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347891098 TSecr=19536021 WS=128

but whenever I try to use something like wget the request fails.

$ wget -T60 google.com
--2015-09-03 11:28:31--  http://google.com/
Resolving google.com (google.com)... 216.58.220.110, 2404:6800:4006:801::200e
Connecting to google.com (google.com)|216.58.220.110|:80... connected.
HTTP request sent, awaiting response... Read error (Connection timed out) in headers.
Retrying.

I'm also getting a strange traceroute response

$ traceroute to google.com (216.58.220.110), 30 hops max, 60 byte packets
 1  Firewall.domain.lan (Removed)  1.002 ms  0.994 ms  0.973 ms
 2  192.168.1.1 (192.168.1.1)  2.018 ms  2.010 ms  1.991 ms
 3  * * *
 4  te1-1-6.sydgscore1.wireline.com.au (103.19.172.61)  103.634 ms  107.541 ms                                     109.968 ms
 5  ge1-1-2.sydgsbdr2.wireline.com.au (103.19.172.34)  111.886 ms  115.329 ms  1                                   17.046 ms
 6  as15169.nsw.ix.asn.au (218.100.52.3)  120.206 ms  121.862 ms  123.631 ms
 7  209.85.242.124 (209.85.242.124)  128.192 ms  147.385 ms  129.917 ms
 8  * * *
 9  * * *
 ...
 29  * * *
 30  * * *

But works fine from windows

$ tracert google.com
Tracing route to google.com [216.58.220.110] over a maximum of 30 hops:

  1     1 ms     2 ms     2 ms  Firewall.domain.lan [Removed]
  2     2 ms     2 ms     2 ms  192.168.1.1
  3    63 ms    62 ms    67 ms  1.1.1.1
  4    99 ms    74 ms    55 ms  te1-1-6.sydgscore1.wireline.com.au [103.19.172.61]
  5    63 ms    51 ms    51 ms  ge1-1-2.sydgsbdr2.wireline.com.au [103.19.172.34]
  6    50 ms    55 ms    52 ms  as15169.nsw.ix.asn.au [218.100.52.3]
  7    51 ms    52 ms    62 ms  209.85.242.124
  8    51 ms    51 ms    52 ms  209.85.142.11
  9    52 ms    52 ms    64 ms  syd10s01-in-f14.1e100.net [216.58.220.110]

Trace complete.

Output of ip addr:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:8d:d6:5a brd ff:ff:ff:ff:ff:ff
    inet IP Removed/24 brd 10.0.100.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe8d:d65a/64 scope link
       valid_lft forever preferred_lft forever

Routes all look fine:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         Firewall.lan    0.0.0.0         UG    0      0        0 eth0
IP Removed      *               255.255.255.0   U     0      0        0 eth0

Netstat on the KVM host

$ netstat -i
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
br0        1500 0   2704662      0      0 0       1815107      0      0      0 BMRU
eth0       1500 0   8757521      9     30 0       9220563      0      0      0 BMRU
eth1       1500 0         0      0      0 0             0      0      0      0 BMU
lo        65536 0   1778126      0      0 0       1778126      0      0      0 LRU
vnet0      1500 0       699      0      0 0          5860      0      0      0 BMRU
vnet2      1500 0      6525      0      0 0        432071      0      0      0 BMRU
vnet3      1500 0   3724358      0      0 0       6696965      0      0      0 BMRU
vnet4      1500 0   6956040      0      0 0       6418627      0      0      0 BMRU
vnet5      1500 0   3766203      0      0 0       2957162      0   1107      0 BMRU
vnet6      1500 0   2946274      0      0 0       4322828      0      0      0 BMRU

ARP Table

$ arp -e
Address                  HWtype  HWaddress           Flags Mask            Iface
IP.IP.IP.10              ether   52:54:00:8d:d6:5a   C                     eth0
IP.IP.IP.5               ether   52:54:00:bc:40:03   C                     eth0
firewall.domain.lan   ether   52:54:00:b8:74:20   C                     eth0

ss -lt | grep 80 returns nothing, so no services which might be interfering with port 80 traffic.

export | grep -i proxy returns nothing as I dont have a proxy configured

IPTables configuration:

$ iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
  • 1
    This looks like a firewall issue somewhere. I'm inclined to think you've got an OUTPUT rule to allow --state NEW traffic, but nothing for --state ESTABLISHED, given that the SYNs are OK, but the ACK doesn't appear to be making it out. – womble Sep 3 '15 at 2:33
  • My iptables are empty. I added an accept all rule to see if it helped but still the same issue. iptables -L -n Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination – Duncan Murray Sep 3 '15 at 2:43
0

It turns out there is an issue with VirtIO's hardware TX checksum offloading capabilities which causes problems with PFSense, which exhibits itself as this partially working connection.

Running $sudo ethtool -K eth0 tx off against each of the virtual interfaces fixed the problem.

https://forum.pfsense.org/index.php?topic=88467.0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.