ssh connection through nginx reverse proxy does not accept private key authetification

Question

I am trying to setup a ssh connection to one of my servers through an nginx reverse proxy server I have. I have read that after 1.9 version such a thing is possible and I have managed to set it up and reach my server, however for some reason the public key is not recognized and it switches to password authentication which is not allowed on that server.

It seems as if the connection reaches the desired server but it doesn't know where to look for the key file. Here is the config I have in the ssh_available folder:

upstream media { # you can use the hostname here
        server 192.168.1.xx:22;
  }

server {
  listen xxxx;
  proxy_pass media; # if you use a hostname then supply that here.
}

I have read several guides and they all stop here but I feel like something is missing from this setup. Can anyone shed some light on this please?

Edit: adding the requested info

For nginx -T:

server {

    listen 443;
    server_name my.server.com;

    ssl_certificate /etc/letsencrypt/live/my.server.certificate/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/my.server.certificate/privkey.pem; # managed by Certbot

    ssl on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers on;


    location /transmission {

      proxy_set_header        Host $host;
      proxy_set_header        X-Real-IP $remote_addr;
      proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_pass_header       X-Transmission-Session-Id;
      proxy_set_header        X-Forwarded-Host   $host;
      proxy_set_header        X-Forwarded-Server $host;
      proxy_set_header        X-Forwarded-Proto $scheme;

      # Fix the "It appears that your reverse proxy set up is broken" error.
      proxy_pass          http://192.168.1.xx:xxx;
      proxy_read_timeout  90;
#      proxy_redirect off;
      proxy_buffering off;

      proxy_redirect      http://192.168.1.xx:xxx/ https://my.server.com;
    }
    location /rpc {
      proxy_pass          http://192.168.1.xx:xxx/transmission/rpc;
    }
  }


# configuration file /etc/nginx/ssh_enabled/ssh.reverse.proxy.conf:
upstream media { # you can use the hostname here
        server 192.168.1.xx:xxxx;
  }

server {
  listen xxxxx;
  proxy_pass media; # if you use a hostname then supply that here.
}

For ssh -vvv <server>:


OpenSSH_8.6p1, OpenSSL 1.1.1k  25 Mar 2021
debug1: Reading configuration data /home/user/.ssh/config
debug1: /home/user/.ssh/config line 50: Applying options for media-remote
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/user/.ssh/known_hosts2'
debug2: resolving "my.server.com" port XXXX
debug3: ssh_connect_direct: entering
debug1: Connecting to my.server.com [xxx.xxx.xxx.xxx] port XXXX.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: Connection established.
debug1: identity file /home/user/.ssh/media type 0
debug1: identity file /home/user/.ssh/media-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10+deb10u2
debug1: compat_banner: match: OpenSSH_7.9p1 Debian-10+deb10u2 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to my.server.com:XXXX as 'user'
debug3: put_host_port: [my.server.com]:XXXX
debug3: record_hostkey: found key type ED25519 in file /home/user/.ssh/known_hosts:37
debug3: load_hostkeys_file: loaded 1 keys from [my.server.com]:XXXX
debug1: load_hostkeys: fopen /home/user/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha1-etm@openssh.com,[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],umac-12[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:cB4LbqTdIBaE08IyeF1+mPMsLVpdu3u/j17mO5qkox8
debug3: put_host_port: [xxx.xxx.xxx.xxx]:XXXX
debug3: put_host_port: [my.server.com]:XXXX
debug3: record_hostkey: found key type ED25519 in file /home/user/.ssh/known_hosts:37
debug3: load_hostkeys_file: loaded 1 keys from [my.server.com]:XXXX
debug1: load_hostkeys: fopen /home/user/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[my.server.com]:XXXX' is known and matches the ED25519 host key.
debug1: Found key in /home/user/.ssh/known_hosts:37
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/user/.ssh/media RSA SHA256:TQDSZC+mv4HWKgVvYAMrDV5aYDV9f5Em0dIV8vFVP/g explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/user/.ssh/media RSA SHA256:TQDSZC+mv4HWKgVvYAMrDV5aYDV9f5Em0dIV8vFVP/g explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
[email protected]'s password:

Please show full nginx configuration by running nginx -T. Please also share output of ssh -vvv <server> from the client when you try to connect. — Tero Kilkanen, May 23, 2021 at 11:42
if I understand correctly, try superuser.com/questions/1135208/… — djdomi, May 23, 2021 at 11:53
@TeroKilkanen I am adding the result of nginx -T to the original post and the ssh -vvv answer. — lucian, May 24, 2021 at 6:04

Tero Kilkanen · Accepted Answer · 2021-05-24 06:46:54Z

1

It is surprising to me that SSH handshake proceeds to that stage when the proxy is implemented using server block, which does HTTP(S) reverse proxying.

Anyway, please try a proper TCP proxy instead:

stream {
    listen ip.address:port;
    proxy_pass media;
}

answered May 24, 2021 at 6:46

Tero Kilkanen

37.1k3 gold badges41 silver badges64 bronze badges

thank you for the answer but can you tell me where do I add this? in the ssh.reverse.proxy.conf file?
– lucian
May 24, 2021 at 6:57
thank you for the answer but can you tell me where do I add this? in the ssh.reverse.proxy.conf file? Do I replace what I have in this file with your solution? In nginx.conf file I already have a stream section: stream{ include /etc/nginx/ssh_enabled/*;}. And ip.address is the reverse proxy address? or the VM I try to reach?
– lucian
May 24, 2021 at 7:09
did it! but it is a bit more than your suggested answer so I will add my own answer if you don't mind! thank you for your help!
– lucian
May 24, 2021 at 7:33

Add a comment |

lucian · Accepted Answer · 2021-05-24 07:40:14Z

The solution for this came from this post. The secret is the ssh port of the server that you need to reach behind the proxy server should be different than the ssh port of the proxy server (in my case both servers had port 22 by default).

Changing the ssh port of the server behind the proxy to a different port (eg. 1234) the ssh.reverse.proxy.conf in the ssh_enabled dir becomes:

upstream media {
  server ip.server.behind.revproxy:1234;
}

server {
  listen 1234;
  proxy_pass media;
}

Hope this helps and thanks for all the help!

Stack Exchange Network

ssh connection through nginx reverse proxy does not accept private key authetification

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
nginx
ssh
.

Hot Network Questions

ssh connection through nginx reverse proxy does not accept private key authetification

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged nginxssh.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
nginx
ssh
.