106

All of a sudden (read: without changing any parameters) my netbsd virtualmachine started acting oddly. The symptoms concern ssh tunneling.

From my laptop I launch:

$ ssh -L 7000:localhost:7000 user@host -N -v

Then, in another shell:

$ irssi -c localhost -p 7000

The ssh debug says:

debug1: Connection to port 7000 forwarding to localhost port 7000 requested.
debug1: channel 2: new [direct-tcpip]
channel 2: open failed: connect failed: Connection refused
debug1: channel 2: free: direct-tcpip: listening port 7000 for localhost port 7000, connect from 127.0.0.1 port 53954, nchannels 3

I tried also with localhost:80 to connect to the (remote) web server, with identical results.

The remote host runs NetBSD:

bash-4.2# uname -a
NetBSD host 5.1_STABLE NetBSD 5.1_STABLE (XEN3PAE_DOMU) #6: Fri Nov  4 16:56:31 MET 2011  root@youll-thank-me-later:/m/obj/m/src/sys/arch/i386/compile/XEN3PAE_DOMU i386

I am a bit lost. I tried running tcpdump on the remote host, and I spotted these 'bad chksum':

09:25:55.823849 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 67, bad cksum 0 (->3cb3)!) 127.0.0.1.54381 > 127.0.0.1.7000: P, cksum 0xfe37 (incorrect (-> 0xa801), 1622402406:1622402421(15) ack 1635127887 win 4096 <nop,nop,timestamp 5002727 5002603>

I tried restarting the ssh daemon to no avail. I haven't rebooted yet - perhaps somebody here can suggest other diagnostics. I think it might either be the virtual network card driver, or somebody rooted our ssh.

Ideas..?

4
  • 1
    For troubleshooting, try $ ssh -L 7000:127.0.0.1:7000 user@host -N -v -v. (You can use "-v" up to 3 times to increase verbosity.) Also, is it possible that ssh was recently updated? Mar 30, 2013 at 23:18
  • The output log I pasted was gathered with -v already.
    – lorenzog
    Mar 31, 2013 at 20:46
  • 1
    You can use -v up to three times to increase verbosity. So you might look at the output of ssh -L 7000... -N -v -v (two v's) or ssh -L 7000... -N -v -v -v. Apr 1, 2013 at 0:39
  • 1
    @MikeSherrill'CatRecall' A shorthand can be used as well: -vvv
    – jnns
    Feb 8, 2018 at 10:44

15 Answers 15

76

Problem solved:

$ ssh -L 7000:127.0.0.1:7000 user@host -N -v -v

...apparently, 'localhost' was not liked by the remote host. Yet, remote /etc/hosts contains:

::1                     localhost localhost.
127.0.0.1               localhost localhost.

while the local network interface is

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33184
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2

Sigh. so much for the bounty of 100rp I put on :)

6
  • 1
    Ah. Well then, I won't bother to write up my comment as an answer. (Look into whether ssh prefers ipv6 addresses on your system.) Apr 1, 2013 at 0:45
  • Well you suggested doubling the -v option, but that did not show anything new.. however, by making me look at the output again after a few days helped to pinpoint the problem. If you want to write up the answer, I'm more than happy to give you the bounty.
    – lorenzog
    Apr 1, 2013 at 8:26
  • 3
    Actually, the important point was replacing "localhost" with "127.0.0.1". The additional "-v" arguments might have been helpful, but they weren't what I was aiming at. Thanks. Apr 1, 2013 at 11:51
  • according to that post on superuser: superuser.com/questions/346971/ssh-tunnel-connection-refused a program configured to listen on a specific address will listen to that specific address
    – jopasserat
    Jan 1, 2014 at 3:09
  • 1
    For me adding leading ":" works so command in your case would look like this: ssh -L :7000:127.0.0.1:7000 user@host -N -v -v
    – valentt
    Apr 12, 2017 at 23:55
28

Although OP's problem has already been solved, I decided to share the solution for my problem, because I got the same error message from ssh and I didn't find any solution on other sites.

In my case I had to connect to the service which listens only on IPv6. I tried:

ssh -f [email protected] -L 51005:127.0.0.1:51005 -N
ssh -f [email protected] -L 51005:localhost:51005 -N

and a few other ways but it didn't work. Any try of connection to http://localhost:51005 causes errors like this: channel 2: open failed: connect failed: Connection refused

The solution is:

ssh -f [email protected] -L 51005:[::1]:51005 -N

IPv6 address must be in square brackets.

2
  • 1
    What if your using a ssh config file? example: "LocalForward localhost:64160 192.168.1.56:3389" Nov 13, 2015 at 0:33
  • 1
    For me adding leading ":" works so command in your case would look like this: ssh -f [email protected] -L :51005:127.0.0.1:51005 -N
    – valentt
    Apr 12, 2017 at 23:54
16
+100

I would first try this.

$ ssh -L 7000:127.0.0.1:7000 user@host -N -v -v

You can use "-v" up to 3 times to increase verbosity.

I think this error message can arise if a firewall blocks port 7000, but you had already ruled that out. (If later readers haven't ruled that out, look at the output of netstat --numeric-ports.)

I think I might have seen this error message a long time ago, when ssh first became aware of IPV6 addresses following an update. I could be wrong about that. If you feel like experimenting, you can try the IPV6 loopback address "0:0:0:0:0:0:0:1" (or "::1").

1
  • 1
    Not a firewall on my side but a bit similar. The opened port was used by a webbrowser and there was an ad-blocker activated in the browser. The ad-blocker apparently managed to cause the above-mentioned error message ("chanel 2 ..."). After switching the ad-blocker off, everything worked.
    – rwoo
    May 29, 2020 at 11:34
4

"...apparently, 'localhost' was not liked by the remote host. Yet, remote /etc/hosts contains:"

Except you were running ssh on the client, so 'localhost' was not liked by your client. The remote /etc/hosts file is for the remote connecting out not incoming connections.

1
  • 1
    that was also confusing to me. When you type localhost in your localmachine, it gets resolved locally
    – Ahmadov
    Jan 31, 2018 at 13:11
4

I encountered this same error while trying to connect to mysql on another server via an ssh tunnel. I found that the bind-address parameter in /etc/my.cnf on the target server was bound to my external ip (dual NIC server) rather than internal, which I had no use for.

When I set bind-address=127.0.0.1, I could successfully use my ssh tunnel as follows:

ssh -N -f -L 3307:127.0.0.1:3306 [email protected]

mysql -h 127.0.0.1 --port=3307 --protocol=TCP -uusername -ppassword
1
  • That worked for me as well. You can only bind MySQL to one address.
    – leeand00
    Nov 23, 2015 at 21:38
4

For me adding leading ":" works so command in your case would look like this:

ssh -L :7000:localhost:7000 user@host -N -v
2
  • Too much time has passed and I can't go back and check, but this looks great.
    – lorenzog
    Apr 13, 2017 at 15:44
  • This was the solution for me! Apr 19, 2021 at 7:32
3

I encountered this error when I was forwarding ports with a full domain name instead of localhost:

ssh -L 5900:host.name.com:5900 x11vnc

The port was being opened only for localhost, so to accept connections with a fully qualified name, I had to add a binding port description:

ssh -L *:5900:host.name.com:5900 x11vnc

which would allow connections from anywhere (so it's not that secure, use it sparingly).

3

Alternative interpretation is, in my case, your typing it wrong.

user@host ~ $ ssh -vvvNL 4444:127.0.0.0.1:4444
...
channel 2: open failed: connect failed: Name or service not known

What happens here is the IP address has one too many zeroes, thus not being a valid address. So ssh treats it as a domain name instead which it can't resolve. Oops!

PS: I supplement this so we have comprehensive list of possible problems when troubleshooting same symptoms.

2

???

channel 2: open failed: connect failed: Connection refused

At user@host there's nothing listening port 7000, that's simple and that's all.

5
  • 1
    That's not true. There is a service running at host:7000. I also tried with other services.
    – lorenzog
    Mar 31, 2013 at 20:43
  • 2
    No, then it would just hangs connecting.
    – RickyA
    Oct 10, 2013 at 9:39
  • 4
    @RickyA: Actually, that's not true. If the port isn't bound the connection will be refused. I got this error from using the wrong internal port (where no service was running), the error went away when I corrected the mistake. poige is right in that if nothing is listening on the port, it will cause the error.
    – erb
    Oct 9, 2015 at 9:42
  • so I was right (;
    – poige
    Jan 31, 2021 at 3:54
  • For some reason the server is stopped. start it. Jun 11, 2021 at 21:56
2

For me, I was trying ssh -L <port>:<remote server IP>:<port> <login>@<remote server IP> when I should have been doing ssh -L <port>:127.0.0.1:<port> <login>@<remote server IP>.

I hope this helps someone!

1
  • I just discovered this as well today. The -L specification is resolved on the remote machine, so localhost or 127.0.0.1 is more appropriate than the actual remote host name. In my case, I had ssh -L 7023:xyz:7022 joe@xyz, and it failed to work on some machines but worked on others. Then I discovered that the machines where it failed had localhost defined to be 127.0.0.1 and xyz was set to a network address like 192.168.1.25 and the service I was connection ONLY allowed localhost - literally localhost address and not the machine address.-L 7023:localhost:7022 worked better Apr 12, 2023 at 21:16
1

I received the same error message:

channel 3: open failed: connect failed: Connection refused

And the cause was human error - me trying to access a different port on the remote host than the one I specified.

Just thought I'll share that, although this is probably not the reason why most of you are experiencing this error.

1
  • In my case: this is exactly what I was doing. Such a stupid mistake but it took this answer to make me check the port. Doh.
    – Ken Sharp
    Apr 19, 2018 at 12:55
1

For the benefit of others who made a silly mistake like I did (one that isn't mentioned in other answers): Make sure your remote machine actually has a process listening on the port that you're trying to tunnel to!

As @Kenster says here:

When you connect to port 8783 on your local system, that connection is tunneled through your ssh link to the ssh server on server.com. From there, the ssh server makes TCP connection to localhost port 8783 and relays data between the tunneled connection and the connection to target of the tunnel. The "connection refused" error is coming from the ssh server on server.com when it tries to make the TCP connection to the target of the tunnel. "Connection refused" means that a connection attempt was rejected. The simplest explanation for the rejection is that, on server.com, there's nothing listening for connections on localhost port 8783. In other words, the server software that you were trying to tunnel to isn't running, or else it is running but it's not listening on that port.

So first check that your remote process is running, and then check that it's actually listening on the port that you're trying to connect to.

On your remote machine you can run any one of these (depending on the packages available on your distribution) to check which ports are being listened to:

sudo lsof -i -P -n | grep LISTEN
sudo netstat -tulpn | grep LISTEN
sudo ss -tulpn | grep LISTEN

If your command is ssh -L -N 7000:127.0.0.1:6000 user@host and you can't see anything listening on 6000, then that's your problem - you need to work out why the process isn't listening on a port, or on the port that you expect.

0

It lacks the IP address --ip=n.n.n.n at the end of the line. You should specify exactly which IP to connect.

0

What worked for me was switching the order of the commands.

So basically -

$ ssh user@host -L 7000:localhost:7000 -N

Strange

0

For whatever reason, my issue was fixed by restarting the AWS EC2 where the mongo instance was. Almost as if it was caused by me trying to connect to it too early and the ports got messed up or something..

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .