16

I am configuring OpenVPN 2.3.6-1 on my Arch Linux server in order to encrypt SMB traffic over the public Internet. When I test the setup on one of my Linux virtual machine clients, I get the error: TLS Error: TLS handshake failed.

I quickly read (OpenVPN on OpenVZ TLS Error: TLS handshake failed (google suggested solutions not helping)) and tried to switch from the default UDP to TCP, but that only caused the client to repeatedly report that the connection timed out. I also tried disabling the cipher and TLS authentication, but that caused the server to fail with Assertion failed at crypto_openssl.c:523. In both instances, the required changes were made to both the client and server configurations.

I have been following the instructions at (https://wiki.archlinux.org/index.php/OpenVPN) to set up OpenVPN and the instructions at (https://wiki.archlinux.org/index.php/Create_a_Public_Key_Infrastructure_Using_the_easy-rsa_Scripts) to create the keys and certificates. The only deviations I have made from these instructions have been specifying my own computers' names and their corresponding key/certificate file names.

See also my original question about securing SMB traffic over the Internet: (Simple encryption for Samba shares)

Can anybody explain how I can solve this issue?

Details:

Server: Arch Linux (up to date) connected directly to gateway via ethernet cable. No iptables.

Client: Arch Linux (up to date) virtual machine on VirtualBox 4.3.28r100309 Windows 8.1 host, bridged network adapter. No iptables. Windows Firewall disabled.

Gateway: Port forwarding for port 1194 enabled, no firewall restrictions.

Here are the configuration files on the server and client, respectively. I created these according to the instructions on the Arch Wiki.

/etc/openvpn/server.conf (Non-comment lines only):

port 1194
proto udp
dev tun
ca /etc/openvpn/ca.crt
cert /etc/openvpn/server-name.crt
key /etc/openvpn/server-name.key
dh /etc/openvpn/dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
keepalive 10 120
tls-auth /etc/openvpn/ta.key 0
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
verb 3

/etc/openvpn/client.conf (Non-comment lines only):

client
dev tun
proto udp
remote [my public IP here] 1194
resolv-retry infinite
nobind
user nobody
group nobody
persist-key
persist-tun
ca /etc/openvpn/ca.crt
cert /etc/openvpn/client-name.crt
key /etc/openvpn/client-name.key
remote-cert-tls server
tls-auth /etc/openvpn/ta.key 1
comp-lzo
verb 3

Here are the outputs of running openvpn on the machines with the above configurations. I started the server first, then the client.

The output of openvpn /etc/openvpn/server.conf on the server:

Thu Jul 30 17:02:53 2015 OpenVPN 2.3.6 x86_64-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6] built on Dec  2 2014
Thu Jul 30 17:02:53 2015 library versions: OpenSSL 1.0.2d 9 Jul 2015, LZO 2.09
Thu Jul 30 17:02:53 2015 NOTE: your local LAN uses the extremely common subnet address 192.168.0.x or 192.168.1.x.  Be aware that this might create routing conflicts if you connect to the VPN server from public locations such as internet cafes that use the same subnet.
Thu Jul 30 17:02:53 2015 Diffie-Hellman initialized with 2048 bit key
Thu Jul 30 17:02:53 2015 Control Channel Authentication: using '/etc/openvpn/ta.key' as a OpenVPN static key file
Thu Jul 30 17:02:53 2015 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Thu Jul 30 17:02:53 2015 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Thu Jul 30 17:02:53 2015 Socket Buffers: R=[212992->131072] S=[212992->131072]
Thu Jul 30 17:02:53 2015 ROUTE_GATEWAY 192.168.0.1/255.255.255.0 IFACE=enp5s0 HWADDR=##:##:##:##:##:##
Thu Jul 30 17:02:53 2015 TUN/TAP device tun0 opened
Thu Jul 30 17:02:53 2015 TUN/TAP TX queue length set to 100
Thu Jul 30 17:02:53 2015 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Thu Jul 30 17:02:53 2015 /usr/bin/ip link set dev tun0 up mtu 1500
Thu Jul 30 17:02:53 2015 /usr/bin/ip addr add dev tun0 local 10.8.0.1 peer 10.8.0.2
Thu Jul 30 17:02:53 2015 /usr/bin/ip route add 10.8.0.0/24 via 10.8.0.2
Thu Jul 30 17:02:53 2015 GID set to nobody
Thu Jul 30 17:02:53 2015 UID set to nobody
Thu Jul 30 17:02:53 2015 UDPv4 link local (bound): [undef]
Thu Jul 30 17:02:53 2015 UDPv4 link remote: [undef]
Thu Jul 30 17:02:53 2015 MULTI: multi_init called, r=256 v=256
Thu Jul 30 17:02:53 2015 IFCONFIG POOL: base=10.8.0.4 size=62, ipv6=0
Thu Jul 30 17:02:53 2015 IFCONFIG POOL LIST
Thu Jul 30 17:02:53 2015 Initialization Sequence Completed

The output of openvpn /etc/openvpn/client.conf on the client:

Thu Jul 30 21:03:02 2015 OpenVPN 2.3.6 x86_64-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6] built on Dec  2 2014
Thu Jul 30 21:03:02 2015 library versions: OpenSSL 1.0.2d 9 Jul 2015, LZO 2.09
Thu Jul 30 21:03:02 2015 WARNING: file '/etc/openvpn/client-name.key' is group or others accessible
Thu Jul 30 21:03:02 2015 WARNING: file '/etc/openvpn/ta.key' is group or others accessible
Thu Jul 30 21:03:02 2015 Control Channel Authentication: using '/etc/openvpn/ta.key' as a OpenVPN static key file
Thu Jul 30 21:03:02 2015 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Thu Jul 30 21:03:02 2015 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Thu Jul 30 21:03:02 2015 Socket Buffers: R=[212992->131072] S=[212992->131072]
Thu Jul 30 21:03:02 2015 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Thu Jul 30 21:03:02 2015 UDPv4 link local: [undef]
Thu Jul 30 21:03:02 2015 UDPv4 link remote: [AF_INET][my public IP here]:1194
Thu Jul 30 21:04:02 2015 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Thu Jul 30 21:04:02 2015 TLS Error: TLS handshake failed
Thu Jul 30 21:04:02 2015 SIGUSR1[soft,tls-error] received, process restarting
Thu Jul 30 21:04:02 2015 Restart pause, 2 second(s)
  • 1
    Your client never gets a response from the server at all. Either you have a firewall you forgot about, or your port forwarding isn't working. – Michael Hampton Jul 31 '15 at 1:02
  • 2
    Do a packet sniff, like: tcpdump -ni eth0 udp and port 1194 on the server and ensure if packets are arriving. If they are there may be problem with firewall dropping packets, if no then most probably there is some problem with port forwarding on the router. You may do it on the router as well. Give a shot and try to use some higher port, it's not common but maybe your ISP messed something up, like port 11194/UDP or 53/UDP. – Michal Sokolowski Jul 31 '15 at 9:55
  • Yep, it was the forwarding. I don't usually work with UDP, so I forgot to change the protocol when creating the rule. I'll post that as the answer. – Kyle Aug 1 '15 at 0:12
  • 3
    If the packets show up in tcpdump on the server, is there a way to ensure that they arrive at openvpn properly? – Joost Dec 22 '15 at 14:48
9

I had this problem as well.

Am using digitalocean provider for my server and the problem was with floating ip feature.

In order to fix that, you have to update openvpn config setting:

local <ip anchor>

ip anchor should be an ip adress gathered from ip addr command, see example: enter image description here

Credits to this post

  • I had this issue as well with a pfsense device. Adding the local <ip address of VPN server> entry in the server configuration fixed it. Note that for TCP this entry was not needed, it was only UDP to give the error. – Vincenzo Pii Jan 9 '17 at 12:28
5

As suggested by Michael Hampton and Michal Sokolowski in the comments on my question, it was a problem with the port forwarding rule I created on my gateway. OpenVPN is configured to use UDP, and I forgot to switch from TCP to UDP on the gateway as I usually don't use that protocol. The forwarding rule now uses UDP, and my VPN is functional.

0

If it appears after updating the OS core. Or the incoming packets show up in tcpdump on the server, but still not works. Try a simple firewall disable/enable. Maybe somebody will help.

sudo ufw disable
sudo ufw enable
0

My current configuration would work on some countries but not others. I am suspecting that my current provider is blocking TLS handshake packet. Solution? Since I am the only one using that VPN I have switched to static key authentication which - in my case - proved to be super fast https://openvpn.net/index.php/open-source/documentation/miscellaneous/78-static-key-mini-howto.html

0

I was getting this problem due to a misconfigured default gateway on the server side. The OpenVPN server was getting the connection attempt from the client but the response was then being lost because it never reached the right router.

  • Do you remember where you had to change this ? Was this in etc/openvpn/server.conf ? – Arthur Attout Aug 9 at 11:54
  • I think something was wrong with the routing tables at the server side. Check with ip route. – lanoxx Aug 10 at 14:12
0

I just had this problem. On checking my .ovpn file I saw that the ?.ddns.net had been changed to an IP address, which is why it didn't connect. I changed the IP back to the ?.ddns.net address and it connected.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.