Ok, this might sound a little confusing but I will try to describe it nonetheless: I have in my LAN two routers that are running a HA-CARP setup connected to two DSL WAN lines. The CARP works as intended: When the one DSL modem stops working, the network traffic is automatically routed over the second, backup line and I appear in the internet with the IP-number of the second modem.
Default gateway for internet access via main DSL is 192.168.32.1 which is the virtual IP for both pfsenses connected via DSL modem on 192.168.13.1 to the internet. A backup connection goes to the second DSL modem on 192.168.23.1.
In case of an outage, CARP manages to route all internet traffic over .23.1 I have tested both, pulling the pfsense router out and disconnecting the DSL modems.
However I somehow can not reach .23.1 from within the LAN in normal operation, means, when .13.1 is serving internet access. Traceroute packets always go over .13.1 into the internet but the router knows, that .23.0/24 is accessed via it's interface for the backup line and on the router I can ping .23.1 just not from the LAN.
Routing table on the router
Internet: Destination Gateway Flags Netif Expire default 192.168.13.1 UGS re0 127.0.0.1 link#4 UH lo0 192.168.13.0/24 link#1 U re0 <- Default WAN 192.168.13.2 link#1 UHS lo0 <- vIP for CARP 192.168.13.3 link#1 UHS lo0 <- Real IP 192.168.23.0/24 link#2 U re1 <- Backup WAN 192.168.23.2 link#2 UHS lo0 <- vIP for CARP 192.168.23.3 link#2 UHS lo0 <- Real IP 192.168.32.0/24 link#3 U re2 <- LAN 192.168.32.1 link#3 UHS lo0 <- vIP for CARP 192.168.32.2 link#3 UHS lo0 <- Real IP
Pinging the backup modem from the .32.1 firewall router:
ping 192.168.23.1 PING 192.168.23.1 (192.168.23.1): 56 data bytes 64 bytes from 192.168.23.1: icmp_seq=0 ttl=64 time=0.477 ms 64 bytes from 192.168.23.1: icmp_seq=1 ttl=64 time=0.509 ms
Traceroute from within LAN sends the packets over default WAN into the internet where they die at my ISP
traceroute 192.168.23.1 traceroute to 192.168.23.1 (192.168.23.1), 30 hops max, 60 byte packets 1 192.168.13.1 (192.168.13.1) 1.101 ms 1.087 ms 1.431 ms 2 <some ISP IP number> (ISP IP number) 15.299 ms 15.883 ms 15.871 ms 3 * * *
I have tried adding a route on the LAN server but that doesn't help either
route add -net 192.168.23.0/24 gw 192.168.32.1 route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.32.1 0.0.0.0 UG 0 0 0 eno1 192.168.23.0 192.168.32.1 255.255.255.0 UG 0 0 0 eno1 192.168.32.0 0.0.0.0 255.255.255.0 U 0 0 0 eno1
Why does the router try to send all packets over the default route when it actually knows the 23.0/24 network and it's interface that connects to that network? I thought the workflow is always like "look into the routing table if you know how to reach a network and if not, use the default route".