We have three physical machines with SmartOS on them in a LAN. Two of them, 10.40.0.30
and 10.40.0.240
are hypervisors with two pfSense-based virtualized gateways on them. These gateways have 10.40.0.1
and 10.40.0.2
addresses, respectively. There is a 10.40.0.3
virtual IP shared between the gateways.
Third SmartOS machine, 10.40.0.221
, is a storage host. The following is an ARP table from it:
[root@68-05-ca-1a-d2-eb ~]# arp -a -n
Net to Media Table: IPv4
Device IP Address Mask Flags Phys Addr
------ -------------------- --------------- -------- ---------------
aggr0 224.0.0.251 255.255.255.255 S 01:00:5e:00:00:fb
aggr0 10.40.0.2 255.255.255.255 f2:b4:a3:21:de:9f
aggr0 10.40.0.3 255.255.255.255 00:00:5e:00:01:02
aggr0 10.40.0.1 255.255.255.255 a2:b1:25:07:ea:97
aggr0 10.40.0.15 255.255.255.255 20:6a:8a:0e:a5:6e
aggr0 10.40.0.30 255.255.255.255 02:08:20:ee:5b:8b
aggr0 10.40.0.240 255.255.255.255 02:08:20:3e:aa:9e
aggr0 224.0.0.22 255.255.255.255 S 01:00:5e:00:00:16
aggr0 10.40.0.221 255.255.255.255 SPLA 68:05:ca:1a:d2:eb
[root@68-05-ca-1a-d2-eb ~]#
As you can see, the ARP table has an entry for the shared virtual IP. The problem is that neither we can ping the shared IP from the storage host nor we see incoming ICMP packets on pfSense virtual machines.
We guess the problem is related to virtualization somehow. We removed all restrictions from a virtualized network card:
root@a0-36-9f-17-e3-fe ~ dladm show-linkprop net1 -z d783afd7-f8e0-48bf-bc9d-b41939e7b8f8
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
net1 autopush rw -- -- --
net1 zone rw d783afd7-f8e0-48bf-bc9d-b41939e7b8f8 -- --
net1 state r- unknown up up,down
net1 mtu rw 1500 1500 1500
net1 secondary-macs rw -- -- --
net1 maxbw rw -- -- --
net1 cpus rw -- -- --
net1 cpus-effective r- 1-2 -- --
net1 pool rw -- -- --
net1 pool-effective r- -- -- --
net1 priority rw high high low,medium,high
net1 tagmode rw vlanonly vlanonly normal,vlanonly
net1 protection rw -- -- mac-nospoof,
restricted,
ip-nospoof,
dhcp-nospoof
net1 promisc-filtered rw off on off,on
net1 allowed-ips rw -- -- --
net1 allowed-dhcp-cids rw -- -- --
net1 rxrings rw -- -- --
net1 rxrings-effective r- -- -- --
net1 txrings rw -- -- --
net1 txrings-effective r- -- -- --
net1 txrings-available r- 0 -- --
net1 rxrings-available r- 0 -- --
net1 rxhwclnt-available r- 0 -- --
net1 txhwclnt-available r- 0 -- --
root@a0-36-9f-17-e3-fe ~
but it did not start to work.
What else can we try?