Investigating a slow VPN connection (Cisco ASA IPSec) to a remote office, I noticed on our firewall a lot of access rule matches:
Denied ICMP type=3, code=4 from *ip_address* on interface outside
I noticed that a traceroute to the remote site included the same IP address, somewhere between our ISP and the ISP the remote site uses.
I'm also seeing a message immediately after before saying
No matching connection for ICMP error mesage: icmp src outside *ip_address* dst identity:*firewall_outside_ip_address* (type 3, code 4) on outside interface. Original IP payload: protocol 50 src *firewall_outside_ip_address* dst *remote_site_ip_address*
Cisco suggest that this may be symptoms of an attack, but I don't think so.
Protocol 50 is ESP, which is part of IPSec. The remote site is connected to HQ via IPSec VPN using Cisco ASA 5505 at the remote end and ASA 5510 at HQ.
ICMP type=3, code=4 means Fragmentation Needed and Don't Fragment was Set.
Setting Don't Fragment is normal for IPSec ESP packets.
I think what is happening is that packets are leaving our ASA 5510 with the default MTU of 1500. When it hits the router with ip_address that router is unable to pass the traffic to the next hop that uses a smaller MTU, thus requiring fragmentation. The router is sending an ICMP packet back as DF is set, but our Firewall is blocking this, not because of an access rule, but because for some reason our ASA 5510 wasn't expecting this ICMP message.
I am trying to figure out whether the problem is with the configuration on our HQ ASA 5510 (although we have another 36 sites all working fine), the remote ASA 5505 (which is configured uniformly with our other remote ASA 5505s) or something in between the two.
What should I do next?
Update As requested here are the ICMP lines from the HQ ASA 5510:
icmp unreachable rate-limit 1 burst-size 1
icmp permit any echo-reply outside
icmp permit any time-exceeded outside
icmp permit any unreachable outside