How to grab either bash history or last x number lines from shell process running on other tty

Question

interesting little dilemma. This is Centos. Last night I logged into the physical console on a tty. I launched a command and let it run overnight. The tty is still logged in, but I am currently logged in remotely via ssh (I am not at the console.)

I'd like to know the exact command parameters I used (I don't remember), so I'd like to see the command I ran on that tty. Since the shell is still running it has not written anything to bash_history - the history for that instance is still in memory.

So, my question is if there is a way to remotely retrieve what I want, perhaps:

(a) sending a signal to the running shell, causing it to dump its bash history (b) examining the environment of the running shell to obtain its history information (c) examining the most recent x # of lines from the tty session so that I can see what I typed

or some other means ....

Answer 1

Ok, I figured it out. This is actually a pretty nifty thing to know, as I can foresee it having many uses.

1. open an ssh session to the remote host. launch a screen or tmux session with a few windows. You can determine the pseudo-device for each window by typing "tty." Let's say we have "/dev/pts/[123]" corresponding to three shells we're running in the screen session

2. determine the pid of the bash process in question (the one you want to redirect input and output for). this process is currently associated with a terminal device such as /dev/tty1

3. from screen window 1, run "gdb -p [pid]" and run the following commands within gdb:

   a. p dup2(open("/dev/pts/2",0),0) # this changes the standard input for the target process

   b. p dup2(open("/dev/pts/3",1),1) # this changes the standard output for the target process

   c. p dup2(open("/dev/pts/3",1,),2) # this changes the the standard err for the target process

   d. detach

   e. quit

in other words, What step 3 does is make Window 2 the standard input, and Window 3 the output.

4.from window 1 (/dev/pts/1), "ls -l /proc/[pid]/fd" to verify the file descriptor changes for the bash process we want to manipulate

5.what you type in window 2 is now fed to two place: the original bash shell associated with this window, and the target process. therefore, from window 2 (/dev/pts/2), type "hhiissttoorryy[return][return]". the reason you have to type everything twice is because the input is divied out to both the current bash shell and the target bash shell. This is because the operating knows there are two sources that are tapping into the keyboard input for /dev/pts/2, and it fairly distributes the characters you type. the first character goes to one destination, the next goes to the second destination, etc. If you had three processes whose stdin was /dev/pts/3, then you would have to type each character three times. The kernel feeds the input characters in round-robin fashion to the recipients.

6.You can get around the above inconvenience by temporarily setting the stdin for the original Window 2 bash shell to some unused device, such as /dev/tty5 or something. this makes the /dev/pts/2 keyboard the standard input for only one process (instead of two). now you can type commands as normal (they just won't be echoed to the screen you're in, they will be echoed to /dev/pts/3 instead.)

7.since you typed 'history' and it was fed to the target process, whose stdout is window 3, switch to window 3 so you can see the command output. now you have the command history for the target bash shell.

8.back to window 1, use gdb again on [pid] to reset the standard in,out,err for the target shell back to their original values (/dev/tty1). And you can also set the Window 2 stdin back to what it should be.

9. If you like, you can clean up the extra file descriptors by removing them with 'exec 3>&-' to remove fd 3 for example

Note that when you run gdb on a running process, it suspends execution of the process in the same manner as SIGSTOP. When you "detach" it, it resumes as in SIGCONT. This allows you to manipulate the file descriptors without adverse consequences.

I did not try it out, but you can probably make the above even easier and more user-friendly by opening a shell, determining its tty or pty, temporarily setting the shell's assigned standard in and out to an unusued device, assigning the tty/pty as standard in/out for the target shell. This way you can use a single screen for all input and output with the target shell.

SUMMARY: with the above procedure, as long as you have a shell for which you have access, you can use it as the standard in/out for any other process/shell on the system. This is useful, for example, if you would like to interact with a shell that is running on the console, but you do not have physical access to the host (or a lights-out solution). This of course can be generalize to any number of situations where you want a specific shell to take control of the input/output for any process of your choosing.

Answer 2

I still face this situation semi-regulary and finally came up with some simpler ways to recover in-memory history commands from my "orphaned" bash sessions:

(note: in each example below, replace PID with the orphaned bash session's process id)

Effectively trigger history -a:

$ gdb -p PID -batch -ex 'call maybe_append_history(get_string_value("HISTFILE"))'

Dump the last 10 history entries to your local terminal (pty)

$ gdb -p PID -batch -ex 'call append_history(10, "'$(tty)'")'

Backup the entire history to a temporary file:

$ gdb -p PID -batch -ex 'call write_history("/tmp/history-backup.txt")'

Notes:

• Tested with bash 4.3 on Ubuntu 16.04.01.
• On my system, I had to invoke using sudo gdb -- even to access my own bash processes.
• If the call is successful then you should see $1 = 0 (if you see other values, they are likely errno's -- like when trying to append to a file that doesn't yet exist you will get $1 = 2 (ENOENT)).
• This approach is based on studying what the bash C code does in response to the history command; eg: builtins/history.def#L203-L212

Answer 3

You might be able to get this information from /proc/${PID}/cmdline.