Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am working on an IPSec VPN solution allowing iPhones / iPads to connect to a Linux server running Gentoo. I have been able to get the VPN functioning as expected using PSK authentication (PSK + Login + Password), but I am having trouble getting the VPN working with certificate authentication (Certificate + Login + Password). I am running only Racoon (IPSEC), without l2tp.

When I try to connect from the iPhone, it sometimes succeeds (rarely, I can't find a pattern as to when). Most of the time, the iPhone fails to connect with "Negotiation with the VPN server failed."

The certifications are generated with easy-rsa (installed with openvpn). As follows:

build-key-server ipsec-server
build-key --pkcs11 mgorbach_mobile_iPhone

Am I missing something with my setup?

path certificate "/etc/racoon/ssl";                                   

remote anonymous {                                                    
    exchange_mode main,aggressive;                                    
    ca_type x509 "ca.crt";                                            
    certificate_type x509 "ipsec_server.crt" "ipsec_server.key";      
    proposal_check claim;                                             
    generate_policy on;                                               
    verify_cert off;                                                  
    nat_traversal on;                                                 
    dpd_delay 20;                                                     
    mode_cfg on;                                                      
    ike_frag on;                                                      
    passive on;                                                       
    my_identifier asn1dn;                                             
    script "/etc/racoon/phase1-up.sh" phase1_up;                      
    script "/etc/racoon/phase1-down.sh" phase1_down;                  
    proposal {                                                        
        encryption_algorithm aes 256;                                 
        hash_algorithm sha1;                                          
        authentication_method xauth_rsa_server;                       
        dh_group 5;                                                   
        lifetime time 3600 sec;                                       
    }                                                                 
}                                                                     

mode_cfg {                                                            
    conf_source local;                                                
    network4 10.0.8.1;                                                
    netmask4 255.255.255.0;                                           
    pool_size 10;                                                     
    auth_source system;                                               
    save_passwd off;                                                  
    split_network include 172.16.1.0/24;                              
    pfs_group 2;                                                      
}                                                                     

sainfo anonymous {                                                    
    pfs_group 5;                                                      
    lifetime time 3600 sec;                                           
    encryption_algorithm aes 256;                                     
    authentication_algorithm hmac_sha1;                               
    compression_algorithm deflate;                                    
}

Logs from Server in the failure case:

2012-02-14 20:41:19: INFO: 172.16.1.102[500] used for NAT-T  
2012-02-14 20:41:19: INFO: 172.16.1.102[500] used as isakmp port (fd=11)  
2012-02-14 20:41:19: INFO: 172.16.1.102[4500] used for NAT-T  
2012-02-14 20:41:19: INFO: 172.16.1.102[4500] used as isakmp port (fd=12)  
2012-02-14 20:41:19: INFO: fe80::20e:c6ff:fe89:24a6%eth2[500] used as isakmp port (fd=13)  
2012-02-14 20:41:19: INFO: fe80::20e:c6ff:fe89:24a6%eth2[4500] used as isakmp port (fd=14)  
2012-02-14 20:41:19: INFO: fe80::20e:c6ff:fe89:24a6%br0[500] used as isakmp port (fd=15)  
2012-02-14 20:41:19: INFO: fe80::20e:c6ff:fe89:24a6%br0[4500] used as isakmp port (fd=16)  
2012-02-14 20:41:19: INFO: fe80::459:22ff:fe33:495e%tap0[500] used as isakmp port (fd=17)  
2012-02-14 20:41:19: INFO: fe80::459:22ff:fe33:495e%tap0[4500] used as isakmp port (fd=18)  
2012-02-14 20:41:56: INFO: respond new phase 1 negotiation: 172.16.1.102[500]  <=>174.252.45.42[5331]  
2012-02-14 20:41:56: INFO: begin Identity Protection mode.  
2012-02-14 20:41:56: INFO: received Vendor ID: RFC 3947  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-08  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-07  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-06  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-05  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-04  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-03  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02  
2012-02-14 20:41:56: INFO: received Vendor ID: draft-ietf-ipsra-isakmp-xauth-06.txt  
2012-02-14 20:41:56: INFO: received Vendor ID: CISCO-UNITY  
2012-02-14 20:41:56: INFO: received Vendor ID: DPD  
2012-02-14 20:41:56: [174.252.45.42] INFO: Selected NAT-T version: RFC 3947  
2012-02-14 20:41:56: INFO: Adding xauth VID payload.  
2012-02-14 20:41:56: [172.16.1.102] INFO: Hashing 172.16.1.102[500] with algo #2  
2012-02-14 20:41:56: INFO: NAT-D payload #0 doesn't match  
2012-02-14 20:41:56: [174.252.45.42] INFO: Hashing 174.252.45.42[5331] with algo #2  
2012-02-14 20:41:56: INFO: NAT-D payload #1 doesn't match  
2012-02-14 20:41:56: INFO: NAT detected: ME PEER  
2012-02-14 20:41:56: [174.252.45.42] INFO: Hashing 174.252.45.42[5331] with algo #2  
2012-02-14 20:41:56: [172.16.1.102] INFO: Hashing 172.16.1.102[500] with algo #2  
2012-02-14 20:41:56: INFO: Adding remote and local NAT-D payloads.  
2012-02-14 20:41:58: INFO: NAT-T: ports changed to: 174.252.45.42[5335]<->172.16.1.102[4500]  
2012-02-14 20:41:58: INFO: KA list add: 172.16.1.102[4500]->174.252.45.42[5335]  
2012-02-14 20:41:58: WARNING: CERT validation disabled by configuration  
2012-02-14 20:41:58: INFO: Sending Xauth request  
2012-02-14 20:41:58: [174.252.45.42] INFO: received INITIAL-CONTACT  
2012-02-14 20:41:58: INFO: ISAKMP-SA established 172.16.1.102[4500]-174.252.45.42[5335] spi:732afbfe416c3732:452dbadff1dceda9  
2012-02-14 20:41:58: INFO: Using port 0  
2012-02-14 20:41:58: INFO: login succeeded for user "mgorbach"  
2012-02-14 20:41:58: WARNING: Ignored attribute INTERNAL_ADDRESS_EXPIRY  
2012-02-14 20:41:58: ERROR: Cannot open "/etc/motd"  
2012-02-14 20:41:58: WARNING: Ignored attribute 28683  
2012-02-14 20:41:58: INFO: unsupported PF_KEY message REGISTER  
2012-02-14 20:41:59: INFO: purging ISAKMP-SA spi=732afbfe416c3732:452dbadff1dceda9:00007bb0.  
2012-02-14 20:41:59: INFO: purged ISAKMP-SA spi=732afbfe416c3732:452dbadff1dceda9:00007bb0.  
2012-02-14 20:41:59: INFO: ISAKMP-SA deleted 172.16.1.102[4500]-174.252.45.42[5335] spi:732afbfe416c3732:452dbadff1dceda9  
2012-02-14 20:41:59: INFO: KA remove: 172.16.1.102[4500]->174.252.45.42[5335]  
2012-02-14 20:41:59: INFO: Released port 0  
2012-02-14 20:41:59: INFO: unsupported PF_KEY message REGISTER  
2012-02-14 20:41:59: INFO: respond new phase 1 negotiation: 172.16.1.102[500]<=>174.252.45.42[5331]  
2012-02-14 20:41:59: INFO: begin Identity Protection mode.  
2012-02-14 20:41:59: INFO: received Vendor ID: RFC 3947  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-08  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-07  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-06  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-05  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-04  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-03  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02  
2012-02-14 20:41:59: INFO: received Vendor ID: draft-ietf-ipsra-isakmp-xauth-06.txt  
2012-02-14 20:41:59: INFO: received Vendor ID: CISCO-UNITY  
2012-02-14 20:41:59: INFO: received Vendor ID: DPD  
2012-02-14 20:41:59: [174.252.45.42] INFO: Selected NAT-T version: RFC 3947  
2012-02-14 20:41:59: INFO: Adding xauth VID payload.  
2012-02-14 20:41:59: [172.16.1.102] INFO: Hashing 172.16.1.102[500] with algo #2  
2012-02-14 20:41:59: INFO: NAT-D payload #0 doesn't match  
2012-02-14 20:41:59: [174.252.45.42] INFO: Hashing 174.252.45.42[5331] with algo #2  
2012-02-14 20:41:59: INFO: NAT-D payload #1 doesn't match  
2012-02-14 20:41:59: INFO: NAT detected: ME PEER  
2012-02-14 20:41:59: [174.252.45.42] INFO: Hashing 174.252.45.42[5331] with algo #2  
2012-02-14 20:41:59: [172.16.1.102] INFO: Hashing 172.16.1.102[500] with algo #2  
2012-02-14 20:41:59: INFO: Adding remote and local NAT-D payloads.  
2012-02-14 20:42:01: INFO: NAT-T: ports changed to: 174.252.45.42[5335]<->172.16.1.102[4500]  
2012-02-14 20:42:01: INFO: KA list add: 172.16.1.102[4500]->174.252.45.42[5335]  
2012-02-14 20:42:01: WARNING: CERT validation disabled by configuration  
2012-02-14 20:42:01: INFO: Sending Xauth request  
2012-02-14 20:42:01: [174.252.45.42] INFO: received INITIAL-CONTACT  
2012-02-14 20:42:01: INFO: ISAKMP-SA established 172.16.1.102[4500]-174.252.45.42[5335] spi:d5a612eed1d76757:ea9806655c9c96c8  
2012-02-14 20:42:16: INFO: purging ISAKMP-SA spi=d5a612eed1d76757:ea9806655c9c96c8.  
2012-02-14 20:42:16: INFO: purged ISAKMP-SA spi=d5a612eed1d76757:ea9806655c9c96c8.  
2012-02-14 20:42:16: INFO: ISAKMP-SA deleted 172.16.1.102[4500]-174.252.45.42[5335] spi:d5a612eed1d76757:ea9806655c9c96c8  
2012-02-14 20:42:16: INFO: KA remove: 172.16.1.102[4500]->174.252.45.42[5335]  
2012-02-14 20:42:16: INFO: unsupported PF_KEY message REGISTER  
share|improve this question
    
Can't really help you, but for debugging I found it good to reduce the lifetimes to 300 seconds. That way the other device doesn't hang too long in a bad state (I think). Is the psk->cert change the only thing you did? –  AndreasM Feb 15 '12 at 8:58
    
Yes, I changed my_identifier to asn1dn from fqdn "MyGroupName". –  Michael Gorbach Feb 15 '12 at 18:32
    
... also changed the authentication_method to xauth_psk_server. –  Michael Gorbach Feb 15 '12 at 18:58
    
Well shouldn't matter in this case, plus cert verification is off too. One more thing: Try generate_policy require and/or generate_policy unique. Helped me when I had several tunnels to the target host. –  AndreasM Feb 15 '12 at 19:17
    
well xauth seems to be fine from the log. –  AndreasM Feb 15 '12 at 19:19
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.