Proxying connections in a QEMU-based VM through reverse ssh tunnel

Question

I have the following setup:

1. A QEMU virtual machine (VM1) running on a server (server a), which with it shares a private network and it uses the server's internet connection to access the internet.
2. A server (server a) that be easily accessed via the internet.
3. A private server (server b) that resides in a private network.

My goal is to be able to connect VM1 to server b in such a way that VM1 can access any services (i.e. a web server) running on server b, as a bonus I would also like to be able to access the private network in which server b resides in.

I created a reverse proxy from server b to server a via the following command: ssh -v -o ServerAliveInterval=15 -i “keychain.pem" -N -R localhost:8888:localhost:443 user@publichost.com

And in the /etc/libvirt/hooks/qemu file I added the following entries to forward VM1's port 443 to server a's port 8888 (the reverse shh tunnel):

/sbin/iptables -D FORWARD -o virbr0 -p tcp -d $VM_IP --dport 443 -j ACCEPT

/sbin/iptables -t nat -D PREROUTING -p tcp --dport 8888 -j DNAT --to $VM_IP:443

Where VM_IP is VM1's internal IP address. However when forwarding from 8888->443 server b seems to refuse the connection:

debug1: connect_next: host localhost ([127.0.0.1]:443) in progress, fd=5

debug1: channel 0: new [127.0.0.1]

debug1: confirm forwarded-tcpip

debug1: channel 0: connection failed: Connection refused

connect_to localhost port 443: failed.

debug1: channel 0: free: 127.0.0.1, nchannels 1

I also tried allowing access to port 443 on server b but it does not seem to help: sudo iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT.

I am not sure why the host is refusing the connections. Interesting if I run a very simple python web on server b at port 8080 and reverse ssh tunnel it: ssh -v -o ServerAliveInterval=15 -i “keychain.pem" -N -R localhost:8888:localhost:8080 user@publichost.com

I can access the webserver from server a via curl -k https://localhost:8080 and I can also share it via GatewayPorts=yes with VM1 via curl https://SERVER_A_IP:8080/. I do not understand why server b is refusing the connection - this there an easier way to do this (server b and VM1 are not easily accessible from the internet).

Answer 1

You can create a Wireguard VPN between Server A and Server B to route traffic from your VM to Server B (and to Server B's private network).

Assumptions

You have:

For the purposes of this answer I'm going to assume the following addresses:

On server A

• Server A public address : 192.168.122.28
• Server A vm network: 172.16.10.0/24
• VM 1 address: 172.16.10.10

On server B

• Private network: 192.168.27.0/24
• Router address on private network: 192.168.27.1
• Router public address: none (or unknown)
• Server B private address: 192.168.27.10
• Server B
• Additional devices on Server B private network:
  • X: 192.168.27.21
  • Y: 192.168.27.22
  • Z: 192.168.27.23

You will need to replace these addresses with ones that are correct for your environment.

---

Creating the VPN

1. Install the wireguard-tools package on both systems. This should provide you with both the wg and wg-quick commands.

2. On both Server A and Server B, generate public and private keys for wireguard:

   mkdir -p /etc/wireguard
   cd /etc/wireguard
   wg genkey > privatekey
   wg pubkey < privatekey > publickey
   

3. Create a wireguard configuration on Server A:

   cat > /etc/wireguard/wg0.conf <<EOF
   [Interface]
   Address = 10.200.10.10/24
   ListenPort = 51820
   PostUp = wg set %i private-key /etc/wireguard/privatekey
   
   [Peer]
   PublicKey = <contents of server B /etc/wireguard/publickey>
   AllowedIPs = 192.168.27.0/24,10.200.10.0/24
   EOF
   

   The 10.200.10.0/24 addresses here and in the next step are the "vpn network". It can be anything you want it to be as long as it doesn't conflict with an existing network.

4. Create a wireguard configuration on Server B:

   cat > /etc/wireguard/wg0.conf <<EOF
   [Interface]
   Address = 10.200.10.11/24
   ListenPort = 51820
   PostUp = wg set %i private-key /etc/wireguard/privatekey
   PostUp = until ping -c1 10.200.10.10; do sleep 1; done
   
   [Peer]
   PublicKey = <contents of server A /etc/wireguard/publickey>
   AllowedIPs = 172.16.10.0/24,10.200.10.0/24
   Endpoint = 192.168.122.28:51820
   EOF
   

   The PostUp = until ping ... command here will cause wg-quick up wg0 to block until it is able to successfully ping the VPN address on Server A. This is necessary because only Server A has a public address, so we need to initiate the VPN connection from Server B (if you try to connect from Server A to Server B first, the VPN won't come up because Server A has now way to connect to Server B).

5. On both Server A and Server B, enable ip forwarding:

   sysctl -w net.ipv4.ip_forward=1
   

6. On both systems, bring up wg0:

   wg-quick up wg0
   

Now, from VM1 (172.16.10.10), we can ping Server B (192.167.27.10). We can access any service hosted on that server.

We have access to all the devices on the same private network on server B, but we have a routing problem: Server A (which is the default gateway for VM1) knows how to reach 192.168.27.0/24, but the devices (other than Server B) on the private network don't know anything about the VPN. So if from VM1 you were to try to ping device X at 192.168.27.21, your packets would arrive correctly at device X, but X would attempt to route its reply via the default router for the network -- which wouldn't know what to do with them, so they would just get dropped. Your solution here is:

• Give each device on the private network a route to 172.16.10.0/24, or
• Configure the default router with a route to 172.16.10.0/24

(In both cases routing via Server B, 192.168.27.10)

That would look something like:

ip route add 172.16.10.0/24 via 192.168.27.10
ip route add 10.200.10.0/24 via 192.168.27.10

With the appropriate routes in place, from VM1 I can reach Server B and any device on Server B's private network that has the appropriate routes (or any device at all if we've added the route to the default gateway).