AWS VPC to Office network with OpenVPN

Question

I have OpenVPN server running in AWS VPC and able to connect on premises Server to AWS with OpenVPN Client.

Able to ping Instances running in AWS VPC. But not able to Ping On premises server from EC2. Here is the diagram.

          (Office Network) 10.24.11.1/24                    (AWS VPC) 10.2.0.0/16
+--------------------------------------------+       +----------------------------------+
|                                            |       |                                  |
|                 OpenVPN Client (Pi)        |       |       OpenVPN Server (Ubuntu)    |
|                                            |       |                                  |
|             +--------------------------+   |       |  +--------------------------+    |
|             |                          |   |       |  |                          |    |
|       eth0  |                          |   |tun0   |  |                          |    |
|      <--+-->|                  10.8.0.2|<--|--+----|->|10.8.0.1       10.2.2.101 |ens5|
|             |                          |   |       |  |                          |    |
|         br0 |10.24.11.15               |   |       |  |                          |    |
|             |                          |   |       |  |                          |    |
|             |       wlan0              |   |       |  |                          |    |
|             +--------------------------+   |       |  +--------------------------+    |
|                       ^                    |       |                                  |
|                       |                    |       |                                  |
|     WiFi Client       |                    |       |                                  |
| +------------------+  |                    |       |                  +----------+    |
| |wlan0:10.24.11.201|  |                    |       |                  | Host 1   |    |
| |                  |--+                    |       |                  |          |    |
| +------------------+                       |       |                  |10.2.1.145|    |
+--------------------------------------------+       |                  +----------+    |
                                                     +----------------------------------+

OpenVPN Server route :

ubuntu@openvpn:~$ ip route
default via 10.2.2.1 dev ens5 proto dhcp src 10.2.2.101 metric 100
10.2.2.0/24 dev ens5 proto kernel scope link src 10.2.2.101
10.2.2.1 dev ens5 proto dhcp scope link src 10.2.2.101 metric 100
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.1
10.24.11.0/24 via 10.8.0.2 dev tun0 proto static src 10.8.0.1

OpenVPN Client route :

pi@accesspoint:~ $ ip route
default via 10.24.11.1 dev br0 src 10.24.11.15 metric 203
10.2.0.0/16 via 10.8.0.1 dev tun0 proto static src 10.8.0.2
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2
10.24.11.0/24 dev br0 proto kernel scope link src 10.24.11.15 metric 203
35.176.225.61 via 10.24.11.1 dev br0

WiFi Client route :

pi@client:~ $ ip route
default via 10.24.11.1 dev wlan0 src 10.24.11.201 metric 303
10.2.0.0/16 via 10.24.11.15 dev wlan0 proto static
10.24.11.0/24 dev wlan0 proto kernel scope link src 10.24.11.201 metric 303

Added route for 10.24.11.0/24 & 10.8.0.1/16 to point to OpenVPN Server private IP in VPC route table.

Also have net.ipv4.ip_forward=1 in /etc/sysctl.conf on OpenVPN server & OpenVPN Client.

I am able to ping any ip of the subnet 10.2.0.0/16 from OpenVPN Client & WiFi Client, but can't ping any ip of the subnet 10.24.11.0/24 from 10.2.0.0/16.

Firewall Rules on OpenVPN server :

ubuntu@openvpn:~$ sudo iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

If I ping IP of OpenVPN server 10.2.2.101 & VPN tunnel ip 10.8.0.1 from Host 1 I see packages with sudo tcpdump -i ens5 -nn icmp

So Added route for 10.24.11.0/24 & 10.8.0.1/16 to point to OpenVPN Server private IP in VPC route table is not working.

After Disabling Source/Destination Checks on OpenVPN Server I was able to ping IP of 10.8.0.0/24 from 10.2.0.0/16. But not 10.24.11.0/24.

When I ping IP of 10.24.11.0/24 from Host 1, I do see packages goint into tunnel on OpenVPN Server sudo tcpdump -i tun0 -nn icmp but not coming out from the tunnel on OpenVPN Client sudo tcpdump -i tun0 -nn icmp

Update

I ran sudo iptables -I INPUT -s {OpenVPN-Server-Public-IP} -j ACCEPT, on OpenVPN Client then tried to ping 10.24.11.15 from Host 1. Able to see packets going into tunnel on OpenVPN Server

ubuntu@openvpn:~$ sudo tcpdump -i tun0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
09:38:42.344238 IP 10.2.2.46 > 10.24.11.15: ICMP echo request, id 12967, seq 203, length 64
09:38:43.344222 IP 10.2.2.46 > 10.24.11.15: ICMP echo request, id 12967, seq 204, length 64
09:38:44.344280 IP 10.2.2.46 > 10.24.11.15: ICMP echo request, id 12967, seq 205, length 64

But No package is coming out of the tunnel on OpenVPN Client server

pi@accesspoint:~ $ sudo tcpdump -i tun0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes

When I ping Host 1 from OpenVPN Client it works :

pi@accesspoint:~ $ ping 10.2.2.46
PING 10.2.2.46 (10.2.2.46) 56(84) bytes of data.
64 bytes from 10.2.2.46: icmp_seq=1 ttl=63 time=72.5 ms
64 bytes from 10.2.2.46: icmp_seq=2 ttl=63 time=72.4 ms
64 bytes from 10.2.2.46: icmp_seq=3 ttl=63 time=72.4 ms

following is the tcpdump :

ubuntu@openvpn:~$ sudo tcpdump -i tun0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
16:47:06.002682 IP 10.8.0.2 > 10.2.2.46: ICMP echo request, id 2807, seq 1, length 64
16:47:06.003034 IP 10.2.2.46 > 10.8.0.2: ICMP echo reply, id 2807, seq 1, length 64
16:47:07.003467 IP 10.8.0.2 > 10.2.2.46: ICMP echo request, id 2807, seq 2, length 64
16:47:07.003799 IP 10.2.2.46 > 10.8.0.2: ICMP echo reply, id 2807, seq 2, length 64
16:47:08.004003 IP 10.8.0.2 > 10.2.2.46: ICMP echo request, id 2807, seq 3, length 64
16:47:08.004353 IP 10.2.2.46 > 10.8.0.2: ICMP echo reply, id 2807, seq 3, length 64

and from Host 1 :

ubuntu@host-1:~$ sudo tcpdump -i eth0 -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:47:06.001635 IP 10.2.2.101 > 10.2.2.46: ICMP echo request, id 2807, seq 1, length 64
20:47:06.001694 IP 10.2.2.46 > 10.2.2.101: ICMP echo reply, id 2807, seq 1, length 64
20:47:07.002398 IP 10.2.2.101 > 10.2.2.46: ICMP echo request, id 2807, seq 2, length 64
20:47:07.002445 IP 10.2.2.46 > 10.2.2.101: ICMP echo reply, id 2807, seq 2, length 64
20:47:08.002915 IP 10.2.2.101 > 10.2.2.46: ICMP echo request, id 2807, seq 3, length 64
20:47:08.002961 IP 10.2.2.46 > 10.2.2.101: ICMP echo reply, id 2807, seq 3, length 64

iptables-save from OpenVPN Server :

ubuntu@openvpn:~$ sudo iptables-save
# Generated by iptables-save v1.6.1 on Tue Aug 13 17:01:18 2019
*filter
:INPUT ACCEPT [83032:54462790]
:FORWARD ACCEPT [3756:403911]
:OUTPUT ACCEPT [84053:7826032]
-A FORWARD -i ens5 -o tun0 -j ACCEPT
-A FORWARD -i ens5 -o tun0 -j ACCEPT
COMMIT
# Completed on Tue Aug 13 17:01:18 2019
# Generated by iptables-save v1.6.1 on Tue Aug 13 17:01:18 2019
*nat
:PREROUTING ACCEPT [2350:165593]
:INPUT ACCEPT [276:21608]
:OUTPUT ACCEPT [12682:1773625]
:POSTROUTING ACCEPT [12672:1773673]
-A POSTROUTING -s 10.8.0.0/24 ! -d 10.8.0.0/24 -j SNAT --to-source 10.2.2.101
COMMIT
# Completed on Tue Aug 13 17:01:18 2019

Answer 1

I am able to ping any ip of the subnet 10.2.0.0/16 from OpenVPN Client & WiFi Client, but can't ping any ip of the subnet 10.24.11.0/24 from 10.2.0.0/16.

That suggests that your routing is correct but most likely you've got issues with firewalls. Make sure that:

• the firewall on ubuntu permits new connections from VPC to Office. For start something like this should do:

  iptables -I FORWARD -i ens5 -o tun0 -j ACCEPT
  

• the firewall on accesspoint permits new connections from tunnel to Wifi.

  iptables -I FORWARD -i tun0 -o br0 -j ACCEPT
  

If it still doesn't work start pinging from the AWS Host 1 to the Wifi client and use tcpdump on both ubuntu and accesspoint to verify where the packets go.

1. Verify that the ping packets are coming from Host 1. If not check the Security Group and EC2 Src/Dst checking

   [aws ubuntu] # tcpdump -i ens5 -nn icmp
   

2. Verify that the packets are going into the OpenVPN tunnel. If not check the ubuntu's iptables firewall.

   [aws ubuntu] # tcpdump -i tun0 -nn icmp
   

3. Verify that the packets are arriving in the tunnel to accesspoint.

   [office accesspoint] # tcpdump -i tun0 -nn icmp
   

4. Verify that they are being forwarded from tunnel to Wifi. If not check the firewall on accesspoint.

   [office accesspoint] # tcpdump -i br0 -nn icmp
   

Update based on additional info...

When I ping IP of 10.24.11.0/24 from Host 1, I do see packages going into tunnel on OpenVPN Server sudo tcpdump -i tun0 -nn icmp but not coming out from the tunnel on OpenVPN Client sudo tcpdump -i tun0 -nn icmp

That's odd. This sounds like the accesspoint's firewall may be blocking the inbound VPN packets from the ubuntu.

Can you try to add this one rule to accesspoint's firewall:

iptables -I INPUT -s {aws-ubuntu-public-ip} -j ACCEPT

Hope that helps :)