Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm trying to create an IPSec encrypted VPN tunnel to a Cisco ASA 5540 from a ubuntu server using openswan.

I've set up VPN connections before to Cisco routers, so this task didn't seem like it would be a big issue, but I've been stumped for days on this one.

I have a connection that I need to make with the following parameters (please note my_left_ip is the network ip on my end, my_left_id is the public ip of my server, my_right_ip is the public ip of the vpn router):

conn my-conn
        type=tunnel
        authby=secret
        auth=esp
        ikelifetime=8h
        keylife=8h
        compress=no
        esp=3des128-md5
        ike=3des-md5-modp1024
        keyexchange=ike
        pfs=no
        forceencaps=yes
        # Left security gateway, subnet behind it, nexthop toward right.
        left=my_left_ip
        leftid=my_left_id
        leftsubnet=my_left_subnet
        leftnexthop=%defaultroute
        # Right security gateway, subnet behind it, nexthop toward left.
        right=my_right_ip
        rightid=my_right_ip
        rightsubnet=my_right_subnet
        rightnexthop=%defaultroute
        # To authorize this connection, but not actually start it,
        # at startup, uncomment this.
        auto=start

and my ipsec.secrets file looks like the following:

my_left_ip my_right_ip : PSK "my_shared_secret"

I've varified on the other end that we indeed have the same Pre-shared Key, and our hashing and encryption algorithms match. But I'm getting the following error:

Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: initiating Main Mode
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: ignoring Vendor ID payload [FRAGMENTATION c0000000]
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: STATE_MAIN_I2: sent MI2, expecting MR2
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: received Vendor ID payload [Cisco-Unity]
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: received Vendor ID payload [XAUTH]
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: ignoring unknown Vendor ID payload [4134667de0a482c8eb1dae0586c309fa]
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: ignoring Vendor ID payload [Cisco VPN 3000 Series]
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Dec 19 20:29:57 server01 pluto[30369]: "my-conn" #229: STATE_MAIN_I3: sent MI3, expecting MR3
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: received Vendor ID payload [Dead Peer Detection]
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: Main mode peer ID is ID_IPV4_ADDR: '10.255.255.252'
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: we require peer to have ID 'my_right_ip', but peer declares '10.255.255.252'
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: sending encrypted notification INVALID_ID_INFORMATION to my_right_ip:500
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: next payload type of ISAKMP Hash Payload has an unknown value: 128
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: malformed payload in packet
Dec 19 20:29:58 server01 pluto[30369]: | payload malformed after IV
Dec 19 20:29:58 server01 pluto[30369]: |   00 ea ff f9  b9 c6 69 1b
Dec 19 20:29:58 server01 pluto[30369]: "my-conn" #229: sending notification PAYLOAD_MALFORMED to my_right_ip:500

Here is the config of the Cisco router

 object-group network DM_INLINE_NETWORK_163
        network-object host right_ip_in_subnet1
        network-object host right_ip_in_subnet2
      access-list OUTSIDE-WAN_24_cryptomap line 1 extended permit ip object-group DM_INLINE_NETWORK_163 host my_left_ip
      access-list nonatinside line 224 extended permit ip object-group DM_INLINE_NETWORK_163 host my_left_ip 
      tunnel-group my_left_id type ipsec-l2l
      tunnel-group my_left_id ipsec-attributes
        pre-shared-key **********
        isakmp keepalive threshold 10 retry 2
      crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac
      crypto map outside_map 24 match address OUTSIDE-WAN_24_cryptomap
      crypto map outside_map 24 set  peer  my_left_id
      crypto map outside_map 24 set  transform-set  ESP-3DES-MD5

      access-list OUTSIDE-WAN_24_cryptomap line 1 remark SPROXIL - USA 
      no crypto map outside_map 24 set  nat-t-disable

What could possibly be cause this? Any help would be greatly appreciated.

share|improve this question
    
Lifetimes match? The ASA default for the IKE lifetime is 86400, so verify that it's set to 28800 to match this config. Can you provide the config and any log entries from the ASA? –  Shane Madden Dec 19 '11 at 17:15
    
I cannot provide the cisco logs because the other network is not in my control and they won't give me the logs. I tried using 86400 for the ikelifetime and it still gave me the same error. –  Tucker Dec 19 '11 at 17:33
    
Indeed, 8h is the ASA default for the IPSec SA lifetime. To clarify the different lifetimes: IKE lifetime is the "Phase 1 SA lifetime" in the Cisco world (in their crypto isakmp settings), which defaults to 24 hours. keylife is actually an alias for salifetime, which is the "Phase 2 SA lifetime" on the Cisco side of things (in their crypto ipsec settings), which defaults to 8 hours - compare against what they have. Oh, and have you compared crypto settings against what they use? Ask them on the 3des... settings for phases 1 and 2 (IKE and ESP), as well as whether they're using PFS. –  Shane Madden Dec 19 '11 at 17:50
    
Thanks, I tried changing the ikelifetime and keylife parameters but I get the same result. I've edited my post to include more details about how the connection is failing. I also reconfirmed the crypto settings for ike and esp and they match the routers settings –  Tucker Dec 19 '11 at 20:32
    
we require peer to have ID 'my_right_ip', but peer declares '10.255.255.252 - do those mismatch? Does the config have literally "my_right_ip" configured there? Additionally, is there any NAT translation at all between the two devices? –  Shane Madden Dec 19 '11 at 20:44
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.