0

We have a frustrating issue and I'm hoping someone here can see what I'm missing. We have a small hub and spoke network between 4 sites and a datacenter, each location connecting to each other location via a site to site VPN. The issue is between site 4 and the datacenter. The tunnel is up and traffic is going across it. From the datacenter to site 4 we can ping, telnet, file share, etc. However, site 4 cannot ping, telnet, or file share, etc. with anything at the datacenter. Site 4 has a Cisco 1841 router and we have no access to the datacenter networking equipment.

Site 4 local network is 192.168.56.0/24 with the external address 77.103.76.150

Datacenter local network is 192.168.48.0/24 with the external address 208.7.247.32

Site 4 router config

version 12.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service sequence-numbers
!
hostname fss_bosjb
!
boot-start-marker
boot-end-marker
!
no logging on
!
aaa new-model
!
!
aaa authentication login default local
aaa authorization network AUTHLIST local
!
!
aaa session-id common
dot11 syslog
no ip source-route
ip cef
!
!
no ip dhcp use vrf connected
ip dhcp excluded-address 192.168.56.1 192.168.56.20
ip dhcp excluded-address 192.168.56.240 192.168.56.254
!
ip dhcp pool POOL1
   network 192.168.56.0 255.255.255.0
   default-router 192.168.56.254
   option 4 ip 192.168.56.254
   option 156 ascii "ftpservers=10.10.30.10"
   dns-server 192.168.16.16 192.168.48.10 8.8.8.8 8.8.4.4
!
!
!
multilink bundle-name authenticated
!
!
!
username __ privilege 15 secret 5 __
username __ privilege 15 password 0 __
archive
 log config
  hidekeys
!
!
crypto isakmp policy 1
 encr 3des
 hash md5
 authentication pre-share
 group 2
!
crypto isakmp policy 2
 encr 3des
 authentication pre-share
 group 2
 lifetime 28800
crypto isakmp key __ address 77.105.85.254 no-xauth
crypto isakmp key __ address 200.228.290.174 no-xauth
crypto isakmp key __ address 77.103.89.168 no-xauth
crypto isakmp key __ address 208.7.247.32
crypto isakmp invalid-spi-recovery
crypto isakmp aggressive-mode disable
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set 3DESMD5 esp-3des esp-md5-hmac
 mode transport
!
crypto ipsec profile VTI
 set security-association lifetime seconds 1800
 set transform-set 3DESMD5
!
!
crypto map CMAP 10 ipsec-isakmp
 set peer 208.7.247.32
 set transform-set ESP-3DES-SHA
 set pfs group2
 match address SINGLEHOP
!
!
!
ip tcp synwait-time 10
!
class-map match-all SHOREQOS
 match access-group name SHOREQOS
 match ip dscp ef
!
!
policy-map SHOREQOS
 class SHOREQOS
  priority 432
 class class-default
  fair-queue
!
!
!
!
interface Tunnel0
 description TO_CLEVELAND
 ip address 12.12.12.2 255.255.255.252
 ip nat inside
 ip virtual-reassembly
 qos pre-classify
 tunnel source 77.103.76.150
 tunnel destination 77.105.85.254
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile VTI
!
interface Tunnel2
 description TO_BOSTON
 ip address 12.12.12.10 255.255.255.252
 ip nat inside
 ip virtual-reassembly
 qos pre-classify
 tunnel source 77.103.76.150
 tunnel destination 77.103.89.168
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile VTI
!
interface Tunnel3
 description TO_DALLAS
 ip address 12.12.12.6 255.255.255.252
 ip nat inside
 ip virtual-reassembly
 qos pre-classify
 tunnel source 77.103.76.150
 tunnel destination 200.228.290.174
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile VTI
!
interface FastEthernet0/0
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nat outside
 ip virtual-reassembly
 shutdown
 duplex auto
 speed auto
 no mop enabled
!
interface FastEthernet0/1
 description inside
 ip address 192.168.56.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 ip route-cache flow
 ip tcp adjust-mss 1260
 duplex auto
 speed auto
 no mop enabled
!
interface Serial0/1/0
 description outside
 no ip address
 ip virtual-reassembly
 encapsulation frame-relay IETF
 no ip mroute-cache
 service-module t1 timeslots 1-24
 service-module t1 fdl both
 frame-relay lmi-type ansi
 service-policy output SHOREQOS
!
interface Serial0/1/0.1 point-to-point
 ip address 77.103.76.150 255.255.255.0
 ip access-group 101 in
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nat outside
 ip virtual-reassembly
 snmp trap link-status
 no cdp enable
 frame-relay interface-dlci 16
 crypto map CMAP
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 Serial0/1/0.1
ip route 10.10.30.0 255.255.255.0 12.12.12.1
ip route 192.168.16.0 255.255.255.0 12.12.12.1
ip route 192.168.26.0 255.255.255.0 12.12.12.5
ip route 192.168.36.0 255.255.255.0 12.12.12.9
ip route 192.168.48.0 255.255.255.0 208.7.247.32
!
ip flow-export source Serial0/1/0.1
ip flow-export version 9
ip flow-export destination 208.7.247.32 2055
!
ip http server
no ip http secure-server
ip nat inside source list 102 interface Serial0/1/0.1 overload
ip nat inside source route-map NAT interface Serial0/1/0.1 overload
!
ip access-list extended NAT
 permit ip 192.168.56.0 0.0.0.255 any
 permit ip 10.10.30.0 0.0.0.255 any
ip access-list extended NONAT
 permit ip 192.168.56.0 0.0.0.255 192.168.48.0 0.0.0.255
 permit ip any 192.168.48.0 0.0.0.255
ip access-list extended SHOREQOS
 permit ip 10.10.30.0 0.0.0.255 192.168.56.0 0.0.0.255
 permit ip 192.168.56.0 0.0.0.255 10.10.30.0 0.0.0.255
 permit tcp any any eq 5004
 permit udp any any eq 5004
 permit udp any any eq 2427
 permit udp any any eq 2727
 permit udp any any range 5440 5446
 permit udp host 10.10.30.10 gt 1024 any gt 1024
ip access-list extended SINGLEHOP
 permit ip 192.168.56.0 0.0.0.255 192.168.48.0 0.0.0.255
!
!
map-class frame-relay mlp
!
map-class frame-relay INET
 frame-relay cir 2918400
 frame-relay mincir 1459200
access-list 1 permit 192.168.56.0 0.0.0.255
access-list 23 permit 192.168.0.0 0.0.255.255
access-list 100 deny   ip host 255.255.255.255 any
access-list 100 deny   ip 127.0.0.0 0.255.255.255 any
access-list 100 permit ip any any
access-list 100 permit ip 192.168.56.0 0.0.0.255 any
access-list 101 permit icmp any host 77.103.76.150 echo-reply
access-list 101 permit icmp any host 77.103.76.150 time-exceeded
access-list 101 permit icmp any host 77.103.76.150 unreachable
access-list 101 permit icmp any any echo-reply
access-list 101 permit icmp any any unreachable
access-list 101 permit ip 10.216.191.0 0.0.0.255 192.168.56.0 0.0.0.255
access-list 101 permit udp host 209.190.176.52 host 77.103.76.150 eq non500-isakmp
access-list 101 permit udp host 209.190.176.52 host 77.103.76.150 eq isakmp
access-list 101 permit esp host 209.190.176.52 host 77.103.76.150
access-list 101 permit ahp host 209.190.176.52 host 77.103.76.150
access-list 101 permit udp host 77.105.85.254 host 77.103.76.150 eq isakmp
access-list 101 permit udp host 77.105.85.254 host 77.103.76.150 eq non500-isakmp
access-list 101 permit esp host 77.105.85.254 host 77.103.76.150
access-list 101 permit tcp host 207.58.230.2 host 77.103.76.150
access-list 101 permit tcp host 207.58.199.66 host 77.103.76.150
access-list 101 permit udp host 207.58.230.2 host 77.103.76.150 eq snmp
access-list 101 permit udp host 207.58.199.66 host 77.103.76.150
access-list 101 permit tcp host 207.58.230.2 host 77.103.76.150 eq 2055
access-list 101 permit icmp host 207.58.230.2 host 77.103.76.150
access-list 101 permit icmp host 207.58.199.66 host 77.103.76.150
access-list 101 permit udp any host 77.103.76.150 eq ntp
access-list 101 permit udp host 200.228.290.174 host 77.103.76.150 eq isakmp
access-list 101 permit udp host 200.228.290.174 host 77.103.76.150 eq non500-isakmp
access-list 101 permit esp host 200.228.290.174 host 77.103.76.150
access-list 101 permit udp host 64.81.160.18 host 77.103.76.150 eq isakmp
access-list 101 permit udp host 64.81.160.18 host 77.103.76.150 eq non500-isakmp
access-list 101 permit udp host 77.103.89.168 host 77.103.76.150 eq isakmp
access-list 101 permit udp host 77.103.89.168 host 77.103.76.150 eq non500-isakmp
access-list 101 permit esp host 77.103.89.168 host 77.103.76.150
access-list 101 permit ip host 77.103.89.168 any
access-list 101 permit udp host 208.7.247.32 host 77.103.76.150 eq isakmp
access-list 101 permit udp host 208.7.247.32 host 77.103.76.150 eq non500-isakmp
access-list 101 permit esp host 208.7.247.32 host 77.103.76.150
access-list 101 permit ip host 208.7.247.32 any
access-list 101 permit icmp any any
access-list 102 deny   ip 192.168.56.0 0.0.0.255 192.168.48.0 0.0.0.255
access-list 102 permit ip 192.168.56.0 0.0.0.255 any
snmp-server community public RO
!
!
route-map NAT deny 10
 match ip address NONAT
!
route-map NAT permit 20
 match ip address NAT
!
!
!
control-plane
!
!
line con 0
 exec-timeout 0 0
 privilege level 15
 transport output telnet
line aux 0
 transport output telnet
line vty 0 4
 access-class 23 in
 exec-timeout 0 0
 privilege level 15
 transport input telnet ssh
line vty 5 15
 access-class 23 in
 exec-timeout 0 0
 privilege level 15
 transport input telnet ssh
!
scheduler allocate 20000 1000
ntp clock-period 17177969
ntp server 10.10.30.10
end

Site 4 crypto isakmp sa output

IPv4 Crypto ISAKMP SA
dst             src             state          conn-id slot status
77.103.76.150    208.7.247.32    QM_IDLE           1021    0 ACTIVE
200.228.290.174 77.103.76.150    QM_IDLE           1015    0 ACTIVE
77.103.89.168    77.103.76.150    QM_IDLE           1019    0 ACTIVE
77.105.85.254    77.103.76.150    QM_IDLE           1020    0 ACTIVE

IPv6 Crypto ISAKMP SA

Site 4 crypto ipsec sa output

interface: Tunnel0
    Crypto map tag: Tunnel0-head-0, local addr 77.103.76.150

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 77.105.85.254 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 3647359, #pkts encrypt: 3647359, #pkts digest: 3647359
    #pkts decaps: 6229930, #pkts decrypt: 6229930, #pkts verify: 6229930
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 9

     local crypto endpt.: 77.103.76.150, remote crypto endpt.: 77.105.85.254
     path mtu 1514, ip mtu 1514, ip mtu idb Tunnel0
     current outbound spi: 0xC5CF72B3(3318706867)

     inbound esp sas:
      spi: 0xF4791294(4101575316)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2099, flow_id: FPGA:99, crypto map: Tunnel0-head-0
        sa timing: remaining key lifetime (k/sec): (4535543/827)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xC5CF72B3(3318706867)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2100, flow_id: FPGA:100, crypto map: Tunnel0-head-0
        sa timing: remaining key lifetime (k/sec): (4541607/827)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

interface: Tunnel2
    Crypto map tag: Tunnel2-head-0, local addr 77.103.76.150

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 77.103.89.168 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 136300, #pkts encrypt: 136300, #pkts digest: 136300
    #pkts decaps: 136080, #pkts decrypt: 136080, #pkts verify: 136080
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 77.103.76.150, remote crypto endpt.: 77.103.89.168
     path mtu 1514, ip mtu 1514, ip mtu idb Tunnel2
     current outbound spi: 0x6D1944E5(1830372581)

     inbound esp sas:
      spi: 0xEDE4F99F(3991206303)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2097, flow_id: FPGA:97, crypto map: Tunnel2-head-0
        sa timing: remaining key lifetime (k/sec): (4590264/813)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x6D1944E5(1830372581)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2098, flow_id: FPGA:98, crypto map: Tunnel2-head-0
        sa timing: remaining key lifetime (k/sec): (4590265/813)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

interface: Tunnel3
    Crypto map tag: Tunnel3-head-0, local addr 77.103.76.150

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 200.228.290.174 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 21, #pkts encrypt: 21, #pkts digest: 21
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 77.103.76.150, remote crypto endpt.: 200.228.290.174
     path mtu 1514, ip mtu 1514, ip mtu idb Tunnel3
     current outbound spi: 0xCED8489F(3470280863)

     inbound esp sas:
      spi: 0xD36E64B7(3547227319)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2101, flow_id: FPGA:101, crypto map: Tunnel3-head-0
        sa timing: remaining key lifetime (k/sec): (4464382/1072)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xCED8489F(3470280863)
        transform: esp-3des esp-md5-hmac ,
        in use settings ={Tunnel, }
        conn id: 2102, flow_id: FPGA:102, crypto map: Tunnel3-head-0
        sa timing: remaining key lifetime (k/sec): (4464382/1072)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

interface: Serial0/1/0.1
    Crypto map tag: CMAP, local addr 77.103.76.150

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (192.168.56.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (192.168.48.0/255.255.255.0/0/0)
   current_peer 208.7.247.32  port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 16230, #pkts encrypt: 16230, #pkts digest: 16230
    #pkts decaps: 4328, #pkts decrypt: 4328, #pkts verify: 4328
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 4, #recv errors 0

     local crypto endpt.: 77.103.76.150, remote crypto endpt.: 208.7.247.32 
     path mtu 1500, ip mtu 1500, ip mtu idb Serial0/1/0.1
     current outbound spi: 0x876495FA(2271516154)

     inbound esp sas:
      spi: 0x924BC9DD(2454440413)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2095, flow_id: FPGA:95, crypto map: CMAP
        sa timing: remaining key lifetime (k/sec): (4515363/1662)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x876495FA(2271516154)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2096, flow_id: FPGA:96, crypto map: CMAP
        sa timing: remaining key lifetime (k/sec): (4515309/1662)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

Any help would be appreciated. If anyone wants to see any other outputs let me know.

0

I'm curious about the configuration of the Tunnels. From your description this router should have a connection to the DC, configured (i'm guessing) under Tunnel 3.

However the config for this tunnel looks like it has the wrong destination set.

interface Tunnel3
 description TO_DALLAS
 ip address 12.12.12.6 255.255.255.252
 ip nat inside
 ip virtual-reassembly
 qos pre-classify
 tunnel source 77.103.76.150
 tunnel destination 200.228.290.174
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile VTI

You stated that the DC has an external IP of 208.7.247.32 - should that not be the tunnel destination? This seems to be further reinforced by the sh crypto output:

IPv4 Crypto ISAKMP SA
dst             src             state          conn-id slot status
77.103.76.150    208.7.247.32    QM_IDLE           1021    0 ACTIVE
200.228.290.174 77.103.76.150    QM_IDLE           1015    0 ACTIVE
77.103.89.168    77.103.76.150    QM_IDLE           1019    0 ACTIVE
77.105.85.254    77.103.76.150    QM_IDLE           1020    0 ACTIVE

Here we can see a tunnel with a src of the DC arriving at your router, but we should see is a tunnel leaving the router for a the DC destination. Instead, as per the config on Tunnel 3, we see a tunnel to a different destination.

So it seems to me that we should see a tunnel with src 77.103.76.150 and dst 208.7.247.32.

I could be way off on this but hopefully some food for thought.

A couple more points for thought:

  • This seems to be a full mesh topology rather than hub and spoke?
  • This seems to be using tunnel protection over an already encrypted link - double encryption?
  • Have you considered using DMVPN and using DC router as a hub for inter-site with backup S2S tunnels if the DC goes dark (not that it should..)?

Hope this helps!

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.