34

Reading this question got me to wondering. Assuming screen is not being used. If an SSH session on a Linux target is dropped, for whatever reason, and you reconnect before the server kills the session because of timeout, is it possible to regain control of the running command such that it will not be aborted because of the broken session?

5
  • What command is it? I'm guessing generally the answer is no.
    – davr
    Feb 23, 2010 at 23:11
  • No particular command, I'm just asking as a general concept. Feb 24, 2010 at 0:16
  • 1
    Someone with a comprehensive understanding as to how tty sessions are initialized might be able to tell us how. Seems like if you could recreate a new session on the same tty and explicitly assign the previous PPID it could be possible. I'm just waiting for some bearded nix guru to come along and blow our minds. That's the dream any way. Feb 24, 2010 at 6:25
  • What happens if you experiment with a tethered laptop and pull out the ethernet cord?
    – Paul
    Feb 24, 2010 at 8:35
  • @~drpaulbrewer - Exactly the same as when you do that with a desktop machine - the connection is dropped. How the connection is dropped is irrelevant to the question. Feb 24, 2010 at 22:13

7 Answers 7

13

Attempting to connect a new terminal's current STD* file descriptors to an old running process is just asking for trouble. Even if you do manage to do that, the terminal's job control won't work as expected. You'll have a mess left behind if you eventually exit the taken-over program, and what happens to the shell that sacrificed its file descriptors to be handed to the newly-backgrounded process. Will ssh stay open when that shell goes away? Probably not. So you'll need to redirect it somewhere else first.

Possible or not, I'd wager that it's more desirable to just let the abandoned process get killed "naturally". If you're doing anything important enough to justify trying to do all the hackery required to resume control and you're on an unstable link, you should probably know that in advance and just use screen (or vnc, or whatever floats your detached-control boat). :)

1
  • I'm finding it hard to pick an answer to accept, so I'm accepting this one because at this time it's the only one that has an upvote. Mar 18, 2010 at 8:23
5

I know this is an old question , but I felt it is important to add my findings in case someone else comes across this like I did.

I haven't seen any unusual consequences to doing this yes, but this is what I used and it worked amazingly. Sometimes when we run long processes on our server it will occasionally disconnect the ssh session. The process along with the tty session appears to stay running but we can't reconnect to it. I found the program below to pull the process to the newly connected session.

https://github.com/nelhage/reptyr

Here's more info

https://blog.nelhage.com/2011/02/changing-ctty/

2
  • 6
    Welcome to Server Fault! Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – EEAA
    Apr 3, 2014 at 20:52
  • thank you, @user215086! Surprisingly, this worked! I was editing a long config file for awhile, added a bunch of custom settings and carefully written comments, and was almost finished when the connection dropped! "Noooooo!!..." After I was done shouting profanity at the ceiling, I installed reptyr, et voila! I recovered the ssh session right where I left it still inside the editor, finished a few things, saved, done!! WooHoo! reptyr is awesome.
    – ColdCold
    Nov 17, 2018 at 18:14
4

Generally, the right way to handle this is to prepare for it ahead of time, using GNU screen or bash's nohup or disown mechanisms. If you are using tcsh, the shell will disown background jobs when it exits abnormally.

If you aren't using screen but have managed to keep your process running via one of the disown methods, you might be able to fake reconnecting to the process with gdb (source):

[...] with some dirty hacks, it is not impossible to reopen a process' stdout/stderr/stdin. [...]

And then use gdb for instance to attach to the process, do some call close(0)
call close(1)
call close(2)
call open("/dev/pts/xx", ...)
call dup(0)
call dup(0)
detach

Now, you'd have to tweak this process for your situation. I doubt it would help if you haven't managed to disown the process. If you're using bash, see this post about making bash automatically disown background processes on exit (basically, turn off huponexit with shopt). With a foreground process, you need to have used nohup.

1

Probably not. I cannot guarantee that it is impossible but I really doubt it.

One thing is the lack of killing the shell and possible commands running as a consequence of the termination of the ssh connection. This is not so difficult, you should be able to use nohup and similar mechanisms like mentioned in the other question.

But then, assume that you started ssh somehost nuhup vim /some/file and the connection dies. You run ssh somehost to log in again and can see that your vim process is still running. But so, how do you connect to that process again? Interactive forground processes have a controlling tty and the one opened for your vim process when it started would since then have been closed. I am not sure if there is any way of "reopening" it again in your new shell (just as if you have several background jobs running in one shell you cannot foreground any of those in another shell).

Screen have explicitly been written to have this functionality. At startup it forks two processes, a terminal management process and and a client process. The interaction is client <--> terminal manager <--> application, and when you detach or lose connection the client process dies while the terminal manager continues to live. Screen have some specific support to attach to the terminal management process again later on, and I do not think this is possible in the general case.

2
  • Thanks. That's pretty much what I expected. Let's see if someone can prove us wrong. ;) Feb 24, 2010 at 0:18
  • I have learned since writing this answer that there is a program reptyr for "re-ptying" processes. So in theory doable, but I still think the overall answer is probably not.
    – hlovdal
    May 30, 2017 at 17:36
1

retty might be able to help you, but the disclaimers are very real and relevant :)

1

If session is droped, it means TTL is already expired, so there is no more tty for you (as I understand it). But, if your network connection is disrupted, your SSH session might not need to go down, and you should be able to resume your connection and continue. Is that what you are asking about?

2
  • I was asking in general but I am most interested in when a network issue causes a connection to drop. Overall, I'd say it's not looking too good. Still, this was asked only out of curiosity, rather than need. It's always nice to know the answer before you need it. ;) Feb 27, 2010 at 8:14
  • a dropped connection is not as a simple concept, as it seems. You might experience several different stages of it. For example, you should be able to unplug your network cable, plug it back in a couple of seconds and your SSH session will stay up, you won't have to reconnect to continue, even though you might experience a short initial delay. If you find yourself with a killed ssh session, it means something is misconfigured (or rather not configured properly to support this kind of a disruption).
    – monomyth
    Mar 1, 2010 at 18:05
0

There was a link to some hacky tty stealing code in this question. You should theoretically be able to use this to regain control of a nohup process.

1
  • 1
    The disown command mentioned in the answers to that question is also worth investigating further. Thanks. Feb 24, 2010 at 1:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .