11

I regularly connect via SSH to a remote server, from an Ubuntu system, on the default port 22. Let's call the server example.org. I am sure that this server is configured properly, and I have confirmed that the following issue is independent from my OS and persists across re-installs.

There is one particular Wifi access point where, if I connect to the server (ssh example.org), I get this:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:[REDACTED].
Please contact your system administrator.
Add correct host key in /home/user/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/user/.ssh/known_hosts:3
  remove with:
  ssh-keygen -f "/home/user/.ssh/known_hosts" -R "serv.org"
ED25519 host key for serv.org has changed and you have requested strict checking.
Host key verification failed.

The problematic access point belongs to an academic institution, and seems to be more locked-down than commercial ISP networks (for example I can't download torrents on it). If I go back to another network (say, using my phone as an access point), I can connect again.

According to Wireshark:

  • The DNS query (to 8.8.8.8) for example.org returns the same IP address, even on the problematic access point.
  • The SSH key exchange seems to happen as usual, but the key sent by the server in the "ECDH Key Exchange Reply" indeed has a different fingerprint when I am connecting through the problematic AP.

I don't understand what this network is doing. Blocking port 22 would be one thing, but here I seem to reach the server and get a wrong key as a response.

Could this access point be intentionally tampering with the SSH connection? Is there a way for me to securely use SSH over it despite this? Should I just avoid using it?

  • 1
    Just FYI: if you need to use an example domain you should really use example.com or example.org. They are the official examples domains. If you use a random domain name it could lead people to actually end up on the site, or maybe the site could receive unwanted attention from your "publicity". – Giacomo Alzetta Nov 18 at 17:42
  • @GiacomoAlzetta I didn’t know that, thank you! – Hey Nov 18 at 18:23
22

Either this systems host keys are changing or someone/something is MITM'ing the SSH connection.

The appropriate course of action is to consider that host as compromised (although its likely not the host itself, rather the connection) unless/until you have an explanation.

You may want to reach out to the system administrator of that AP and advise them of your concerns and try and track this down with them.

18

Since it's an academic institution, I'm thinking it's probably a firewall/security gateway that intercepts and analyzes the SSH traffic for things like Data Loss Prevention (DLP) or general audit logging.

For example of such a firewall: https://docs.paloaltonetworks.com/pan-os/9-0/pan-os-admin/decryption/decryption-concepts/ssh-proxy.html

  • 5
    That’s probably what is happening here. On a side note, changing the ssh port is often enough to no get mitm by the so-called firewall. A quick check can be done trying to access github on 443 (intended for over restrictive corporate firewall) ssh -T -p443 git@ssh.github.com – Cyrbil Nov 17 at 10:10
  • I can see this happening on a corporate network, but DLP on a student Wi-Fi?... – grawity Nov 18 at 12:27
  • @Cyrbil Good idea on changing the port. Interesting they would only use simple port-based interception and not do DPI and block SSH not using the standard port (if that's possible). – Rob Olmos Nov 18 at 20:52
  • @grawity It's odd to me it's being done at all and it's bypassed by just changing the port. Maybe it's MITM malware (soft or hard). – Rob Olmos Nov 18 at 20:58
4

The answer by davidgo is correct. Note that, as long as you're using pubkey auth and not password, this MITM does not compromise the safety of your credentials. But it does compromise the privacy and authenticity of anything transmitted over the session. Note that "authenticity of anything transmitted over the session" includes (rather, probably consists entirely of) commands sent to the shell or whatever process you're interacting with and output sent back.

I had originally suggested the following, which is dangerously wrong, at least without further measures:

One easy workaround is double-ssh. Assuming the host you're logging into allows port forwarding, login once with the compromised connection, using pubkey auth and with forwarding enabled to forward a local port to localhost:22, e.g. -L 12345:localhost:22. Now, you can run a second ssh session tunneled through the first, and unless the attacker is actively anticipating you doing this, this second session will have the right host fingerprint and you know that it's actually secure, not subject to interception by the MITM.

This was written hastily as a simplification from the right way I had in mind to do it, which is to have a separate login account used purely for forwarding, with no shell, which I believe is still safe to use with a compromised (MITM'd) connection, but I'd still advise caution. It may also be possible to setup your authorized_keys file to limit the key you use for the initial (MITM'd) login so that it's unable to issue any commands, only forward connections; if so, such a setup should be safe.

  • 1
    If it compromises the authenticity of anything sent over the session, then even with your double-ssh trick, couldn't the attacker take over your outer session and take control of your target box through it? – Joseph Sible-Reinstate Monica Nov 18 at 3:50
  • 1
    @vikarjramun It's not just about blocking your connection. I mean wouldn't it let an attacker completely own your target box? – Joseph Sible-Reinstate Monica Nov 18 at 5:20
  • 1
    @vikarjramun that's my point. Just accepting the bad host key at all seems like a bad idea. And you're not sure whether it's "just" the school or not. – Joseph Sible-Reinstate Monica Nov 18 at 5:55
  • 1
    @JosephSible I guess the idea is to open a broken outer connection, accepting (once) the MITM key, then pipe a secure connection with strict host key verification through it? If strict host key checking was disabled for the outer connection, but enabled for the inner one (and the correct host key is already in our known_hosts, as in my case), then the inner connection will either be secure or fail. Unless I missed something. – Hey Nov 18 at 10:10
  • 2
    @Hey Maybe the inner connection succeeds and is secure, but the MITM still uses your outer connection to install a keylogger. Is that a success, or a failure? – user253751 Nov 18 at 10:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.