1

experiencing some weird issues when trying to setup a new set of routers.

We have single wan coming into two pfsense boxes, with CARP between them on its own interface, and shared Public IP along with a public IP assigned to each box, and on the lan side a shared LAN IP, and then a lan IP assigned to each box, nothing too "complicated" as far as I'm aware.

If I login to the router directly, I can ping any site with no loss - google.com, slashdot, my own home servers. However if I connect from a system on the lan side I'm getting anything between 15 - 60% packet loss. However pinging the router directly with no issues.

Sample of the pings can be seen here: http://pastebin.com/rHqDHv1R

I've disabled all services other than NTPsync and the DNS forwarder.

I'm kind of at a complete loss. The network its self is a bit more complicated - As this is a new set of routers we are doing, there is also an existing pfsense box, which everyone is using currently as their gateway (so I can't turn this off/remove it). However I don't see why this would have any effect on the new systems.

Your help is very much appreciated

As requested, as best as I can produce, a graph showing the current network - Its far larger than pictured with lots more subnets etc however this shows the scope of the current testing and shows where the loss appears.

http://www.gliffy.com/pubdoc/2948337/L.png

Improve this question
4
  • Need some more details on your network setup. A drawing with some fake IPs would help. – gravyface Sep 30 '11 at 10:41
  • Do the two routers also have unique public IP addresses? To do a CARP setup with WAN it takes 3 public IP addresses. A unique public IP address for each router and a shared CARP address that is actually used. What device is upstream of the pfSense boxes any possibility of doing a packet capture there? – 3dinfluence Sep 30 '11 at 13:39
  • Unfortunately not - ISP equipment however yes, we have 3 public IPs, and they are working correctly apart from this intermittent loss. – djsmiley2kStaysInside Sep 30 '11 at 14:31
  • I've never set up CARP myself only looked into it so that's about s much help as I'm going to be :). But your image doesn't show what version of pfSense the new routers are running. I'm assuming it's 2.0 but can you confirm that? Also do the logs show anything? Like CARP status changes etc? – 3dinfluence Sep 30 '11 at 15:52
1

Get a packet capture on LAN and see what's happening. Is the lost traffic making it to the LAN side? My guess is it's not, most likely causes would be either an IP conflict or a MAC conflict (such as using the same CARP VHID on a different pair).

Improve this answer
3
  • Hummm Going to have to capture on teh wan side. Fun times seeing the rubbish thats hitting the box at least :D – djsmiley2kStaysInside Oct 4 '11 at 15:14
  • Rebuilt both boxes, confirmed there is no lag without the CARP/Failover configuration at least. Its possibly the 2nd lan port as the first one works fine either standing alone or in CARP setup. However I'm doubtful about this as I did test it when building and it was ok. – djsmiley2kStaysInside Oct 4 '11 at 15:20
  • Well after lots of testing I'm starting to think there is some bug in either pfsense, or in the switch. Basically after the switch drops the arp record for the pfsense boxes to the "CARP" IP all the traffic hitting the switch seems to disappear into no where... i.e. they never come back.... something upstream is dropping them for some unknown reason - yet after about 10-15min of pinging outwards eventually it "works" again and then is solid working. It seems as long as its in use, it'll work :S – djsmiley2kStaysInside Oct 10 '11 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.