Bash script to send all Wifi hotspot traffic over separate VPN tunnel on Ubuntu 18

Question

I'm working on a bash script that is supposed to create a hotspot on ubuntu 18. However I have special needs about what connection that hotspot is supposed to use.

I want all and any traffic from devices connected to my hotspot to go over a separate VPN tunnel and (not sure this part is relevant) I will randomize which VPN gets chosen from a set of options each time I activate the hotspot.

By default, when creating a hotspot on Ubuntu 18, it will simply share the existing connection whatever it currently happens to be.

When I open nm-connection-editor the GUI contains a VPN option, but I don't know how to create and modify a Hotspot by purely using bash.

I'm basically stuck not knowing how to make it happen programatically or if I lack some other knowledge to achieve this. Unless I missed something there, man pages for nmcli haven't been helpful to me.

What probably complicates things is that I'm already connected to a VPN on tun0, so I'm guessing I need to create a separate VPN connection, let's say over tun1 and make the hotspot use it.

Here is what I have so far

#that's the kind of call I use to activate my main VPN connection
#you can assume I have script already that gives me 
#the value for mainconnectionuuid
nmcli con up uuid mainconnectionuuid #activates tun0, all traffic goes through tun0 after this

#now I want to activate a hotspot that uses a different VPN connection 
#the following code activates the hotspot, but there is no option
#to separate the traffic to go over a different VPN
nmcli radio wifi on
nmcli device wifi hotspot ssid somessid password somepassword

What I need is something like this:

#activate main connection on tun0
nmcli con up uuid mainconnectionuuid

nmcli radio wifi on
#'useconnection' is a phantasy parameter here, but I need something like this
#assuming here that the connection under vpnforhotspotuuid would already define tun1 to be used
nmcli device wifi hotspot ssid somessid password somepassword **useconnection vpnforhotspotuuid**

Or like this:

#activate main connection on tun0
nmcli con up uuid mainconnectionuuid

#activate connection for hotspot on tun1
nmcli con up uuid vpnforhotspotuuid

nmcli radio wifi on
#'usetunnel' is a phantasy parameter here
nmcli device wifi hotspot ssid somessid password somepassword **usetunnel tun1**

How can I achieve this?

I'm not a major expert, I can write bash scripts and I know ufw but I'm somewhat scared of IPtables. I would appreciate a solution that is not too hard to understand as I also want to be able to write a script that undoes or at least deactivates the changes.

EDIT 1:

I tried the proposed solution by Piotr and it seems to almost work, but something goes wrong. When I connect to that hotspot with an android 9 device it seems to work until I execute the last command ip route add default via $TUN1IP table hotspot. After issuing that, I just have just enough time to request whatismyip.com with a browser on the phone and verify that I indeed have the tun1 ip. Short time later (maybe about 15-30 s), the phone wifi settings will start saying "no internet connection" and the phone starts reconnecting in an endless loop until I undo the routing with ip route del default table hotspot

Frist I wondered if this was some kind of firewall issue as my configuration is very strict and starts with

ufw default deny incoming 
ufw default deny outgoing

and then I have some rules to allow vpn connections and local addresses, then regarding the hotspot so far I settled on:

ufw allow out on tun1 to 0.0.0.0/0
ufw allow out on wlan0 to 10.57.213.1/24

Edit 2

Before issuing the final command ip route add default via $TUN1IP table hotspot I see following UFW block (via dmesg), but the connection works anyways though:

[UFW BLOCK] IN=wlan0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx SRC=10.57.213.181 DST=10.57.213.1 LEN=28 TOS=0x00 PREC=0x00 TTL=1 ID=xxxxx DF PROTO=UDP SPT=45104 DPT=4886 LEN=8

Those blocks disappear after issuing the routing command, so I'm guessing they are not significant, not sure though. However if I look at non-ufw messages I see following messages on each reconnection:

netlink: 'wpa_supplicant': attribute type 213 has an invalid length.

I googled a little and some people say it's a bug related to WPA2 and that the connection will work if I disable encryption, but even if I disable encryption nothing changes. The device is still reconnecting to the hotspot in an endless loop.

I should probably also mention that I tried ip route flush cache but it changes nothing

Edit 3

wpa_cli -i wlan0 log_level DEBUG and subsequent journalctl -u wpa_supplicant -e yields (I censored PID and MAC adresses using letter x):

Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-70 fc=0x40 seq_ctrl=0xd740 stype=4
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-72 fc=0x40 seq_ctrl=0xd750 stype=4
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:22 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:33 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:33 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=20:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-75 fc=0x40 seq_ctrl=0x8830 stype=4
Mar 05 20:59:33 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:33 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-72 fc=0x40 seq_ctrl=0xe430 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-70 fc=0x40 seq_ctrl=0xe450 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-71 fc=0x40 seq_ctrl=0xe470 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-71 fc=0x40 seq_ctrl=0xe480 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-71 fc=0x40 seq_ctrl=0xe490 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: BSS Event 59 (NL80211_CMD_FRAME) received for wlan0
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: nl80211: RX frame da=ff:ff:ff:ff:ff:ff sa=10:xx:xx:xx:xx:xx bssid=ff:ff:ff:ff:ff:ff freq=2412 ssi_signal=-71 fc=0x40 seq_ctrl=0xe4a0 stype=4
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: wlan0: Event RX_MGMT (19) received
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: P2P: Not a P2P probe - ignore it
Mar 05 20:59:41 machineone wpa_supplicant[xxxx]: Ignore Probe Request due to DS Params mismatch: chan=1 != ds.chan=2

Edit 4

After activation of tun1 my routing table looks like this

Destination       Gateway           Genmask         Flags Metric Ref    Use Iface
0.0.0.0           10.10.2.1         0.0.0.0         UG    50     0        0 tun0
0.0.0.0           10.10.3.1         0.0.0.0         UG    50     0        0 tun1
0.0.0.0           192.168.xxx.xxx   0.0.0.0         UG    100    0        0 eth0
10.10.2.0         0.0.0.0           255.255.255.0   U     50     0        0 tun0
10.10.3.0         0.0.0.0           255.255.255.0   U     50     0        0 tun1
10.57.213.1       0.0.0.0           255.255.255.0   U     600    0        0 wlan0 
the.vpn.ip.tun0   192.168.xxx.xxx   255.255.255.255 UGH   100    0        0 eth0
169.254.0.0       0.0.0.0           255.255.0.0     U     1000   0        0 virbr0 
the.vpn.ip.tun1   192.168.xxx.xxx   255.255.255.255 UGH   100    0        0 eth0
192.168.yyy.0     0.0.0.0           255.255.255.0   U     0      0        0 virbr0
192.168.xxx.0     0.0.0.0           255.255.255.0   U     100    0        0 eth0
192.168.xxx.xxx   0.0.0.0           255.255.255.255 UH    100    0        0 eth0

I experimented with commands such as ip route add default dev wlan0 via default dev tun1 and ip route add default dev wlp8s0 via 10.10.3.1 dev tun1 but nothing worked. Additionally, sometimes the default gateway for tun0 happens to be the same as for tun1.

edit 5

I discovered there is a bug in the linux kernel (or rather wpa2_supplicant) where the kernel is extremely strict with arguments passed to it and makes wpa2_supplicant fail. I did the following

# nmcli connection edit hotspot
set 802-11-wireless-security.pmf disable
save
[CTRL+D]

This disables Protected Management Frames (PMF) for the hotspot connection I created which is where the kernel call happens that breaks wpa_supplicant. After having done this (and route add default dev wlan0 via 10.10.3.1 dev tun1) now my whole system (not just the hotspot) is running over tun1 instead of tun0. I still need to understand why I can't make it happen that tun0 would be used for eth0 while at the same time tun1 would be used for wlan0.

After I issue the following commands

ip route add default dev wlan0 via 10.10.2.1 dev tun1 proto static metric 50
ip route add default dev eth0 via 10.10.3.1 dev tun0 proto static metric 50

My routing table looks like this:

0.0.0.0          10.10.2.1        0.0.0.0         UG    50     0        0 tun1
0.0.0.0          10.10.3.1        0.0.0.0         UG    50     0        0 tun0
0.0.0.0          192.168.xxx.xxx  0.0.0.0         UG    100    0        0 eth0
10.10.2.0        0.0.0.0          255.255.255.0   U     50     0        0 tun0
10.10.3.0        0.0.0.0          255.255.255.0   U     50     0        0 tun1
10.55.213.1      0.0.0.0          255.255.255.0   U     600    0        0 wlan0
the.vpn.ip.tun1  192.168.xxx.xxx  255.255.255.255 UGH   100    0        0 eth0
169.254.0.0      0.0.0.0          255.255.0.0     U     1000   0        0 virbr0
the.vpn.ip.tun1  192.168.xxx.xxx  255.255.255.255 UGH   100    0        0 eth0
192.168.yyy.0    0.0.0.0          255.255.255.0   U     0      0        0 virbr0
192.168.xxx.0    0.0.0.0          255.255.255.0   U     100    0        0 eth0
192.168.xxx.xxx  0.0.0.0          255.255.255.255 UH    100    0        0 eth0

which looks the same as if I never issued them

edit 6:

when I try to make the vpn connection from the terminal

sudo openvpn --config myconfigfile.ovpn --dev tun1 --verb 3

following route commands get issued and everything will go through tun1 and tun0 is ignored:

Wed Mar 11 19:16:12 2020 /sbin/ip link set dev tun1 up mtu 1500
Wed Mar 11 19:16:12 2020 /sbin/ip addr add dev tun1 10.10.2.6/24 broadcast 10.10.2.255
Wed Mar 11 19:16:12 2020 /sbin/ip route add the.vpn.ip.tun1/32 via 10.10.3.1
Wed Mar 11 19:16:12 2020 /sbin/ip route add 0.0.0.0/1 via 10.10.2.1
Wed Mar 11 19:16:12 2020 /sbin/ip route add 128.0.0.0/1 via 10.10.2.1

I also find the 10.10.3.1 ip odd since it's related to tun0.

When I do

ip route delete 0.0.0.0/1 via 10.10.2.1

all my connections return to tun0 and ignore tun1. ip route show table looks like this:

default via 10.10.3.1 dev tun0 
default via 10.10.3.1 dev tun0 proto static metric 50 
default via 192.168.xxx.xxx dev eth0 proto dhcp metric 100 
10.10.2.0/24 dev tun1 proto kernel scope link src 10.10.2.3 
10.10.3.0/24 dev tun0 proto kernel scope link src 10.10.3.4 metric 50 
10.55.213.0/24 dev wlan0 proto kernel scope link src 10.55.213.1 metric 600 
128.0.0.0/1 via 10.10.2.1 dev tun1 
the.tun0.vpn.ip via 192.168.xxx.xxx dev eth0 proto static metric 100 
169.254.0.0/16 dev virbr0 scope link metric 1000 linkdown 
the.tun1.vpn.ip via 10.10.3.1 dev tun0 
192.168.aaa.bbb/24 dev virbr0 proto kernel scope link src 192.168.aaa.ccc linkdown 
192.168.xxx.0/24 dev eth0 proto kernel scope link src 192.168.xxx.yyy metric 100 
192.168.xxx.xxx dev eth0 proto static scope link metric 100 
broadcast 10.10.2.0 dev tun1 table local proto kernel scope link src 10.10.2.3 
local 10.10.2.3 dev tun1 table local proto kernel scope host src 10.10.2.3 
broadcast 10.10.2.255 dev tun1 table local proto kernel scope link src 10.10.2.3 
broadcast 10.10.3.0 dev tun0 table local proto kernel scope link src 10.10.3.4 
local 10.10.3.4 dev tun0 table local proto kernel scope host src 10.10.3.4 
broadcast 10.10.3.255 dev tun0 table local proto kernel scope link src 10.10.3.4 
broadcast 10.55.213.0 dev wlan0 table local proto kernel scope link src 10.55.213.1 
local 10.55.213.1 dev wlan0 table local proto kernel scope host src 10.55.213.1 
broadcast 10.55.213.255 dev wlan0 table local proto kernel scope link src 10.55.213.1 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.aaa.bbb dev virbr0 table local proto kernel scope link src 192.168.aaa.ccc linkdown 
local 192.168.aaa.ccc dev virbr0 table local proto kernel scope host src 192.168.aaa.ccc 
broadcast 192.168.aaa.255 dev virbr0 table local proto kernel scope link src 192.168.aaa.ccc linkdown 
broadcast 192.168.xxx.0 dev eth0 table local proto kernel scope link src 192.168.xxx.yyy 
local 192.168.xxx.yyy dev eth0 table local proto kernel scope host src 192.168.xxx.yyy 
broadcast 192.168.xxx.255 dev eth0 table local proto kernel scope link src 192.168.xxx.yyy

I hope I edited all those ip's correctly as in reality some ip's change each time, so I'm matching them with my previous edits.

My poor intuition told me to try next

ip route replace 10.57.213.1/24 via 10.10.2.6 dev tun1 table AirSpot

But all I get is Error: Invalid prefix for given prefix length. and I don't understand what it means. I'm just not getting something. Routing is not exactly intuitive to me. With that line I want that any packets addressed from or to 10.57.213.1/24 (which is the manually set hotspot range) are to go through 10.10.2.6, which I would think (based on the verbose output of openvpn) is where the tun1 is.

Edit 7:

I think this is some kind of default route issue, but I'm not expert enough to solve it..

Answer 1

The solution you posted in a comment is almost correct. What is missing is the correct IP addresses to use.

Step 1: Retrieve information

You need to retrieve two pieces of information:

1. The IP addresses of the subnetwork of your hotspot. By default these are a random network of the form 10.42.*.0/24, so you need to fix the addresses. The easiest way is to create a new WiFi connection called hotspot using nm-connection-editor. You need to:

   • set the mode to Hotspot in the "Wi-Fi" tab,
   • set the method to Shared with other computers in the "IPv4 Settings" tab,
   • choose an subnet for the network by adding on IP address (e.g. 10.57.213.1 with netmask 24) in the same tab,
   • add some authentication method (e.g. a PSK)

   After performing these steps, you should have a well configured /etc/NetworkManager/system-connections/hotspot.nmconnection file with a content like:

   [wifi]
   mode=ap
   
   [ipv4]
   address1=10.57.213.1/24
   method=shared
   

   and you can start the connection with nmcli connection up id hotspot.

2. The IP address of the VPN gateway. Whether this is fixed or variable depends on what kind of VPN you are using. Let's say it's 10.10.10.1.

Step 2: Configure policy routing

1. A one-time operation. We need an additional routing table. In order to be able to refer to it by name let's add:

   200 hotspot
   

   to /etc/iproute2/rt_tables.

2. Once per system boot. We need to tell the kernel, that packets originating from 10.57.213.0/24 should use the hotspot table:

   ip rule add from 10.57.213.0/24 table hotspot
   

3. Every time we connect to a VPN, we need to add a default route for the HotSpot addresses:

   ip route replace default via 10.10.3.1 dev tun1 table hotspot
   

   where 10.10.3.1 is the address of the other side of your VPN tunnel according to the routing table.

   You can run this part in a NetworkManager dispatcher script, in which case the gateway address will be in the environment variable $IP4_GATEWAY.

Edit: To automatically establish the default route for the hotspot table add a dispatcher script /etc/NetworkManager/dispatcher.d/hotspot.sh like this:

#!/bin/bash
# Be verbose until you test this
set -ex

# First parameter passed by NM: interface
DEVICE="$1"
# Second parameter: action
CMD="$2"
IP=/bin/ip

# Delete the part after '/' from $IP4_ADDRESS_0
LOCAL_IP=${IP4_ADDRESS_0%/*}
if [ "$LOCAL_IP" != "" ]; then
  # Add a 'src' option to the route only if the IP is not empty
  SRC="src $LOCAL_IP"
fi

if [ "$CMD" = "vpn-up" ]; then
  $IP route replace default via $IP4_GATEWAY dev $DEVICE $SRC table hotspot
fi

I didn't test this.