NOTE: I'm really looking for an SSH-y solution here, more than bash-y one.

I need to execute a series of commands over ssh, programatically. There are many ways to do this, but I can't find any that works with my constraints:

  • some of the commands set variables in the shell. Therefore I dont think I can make this work using multiple shells.

  • some of these variables are functions, not normal values, so I dont think I can simply serialize the output of env to "restore" the old environment.

  • I need to access the stdout, stderr and exit code of each command separately. This makes it difficult to chain the commands using && or ;. I believe this also makes it difficult to use screen, because we cant reliably tell when a command finishes.

  • I need real-time access to the stdout, stderr, etc so we can record when the output occurs. (I use "real-time" to mean "nowish", not "hard real-time" or anything like that).


Why do I need a single bash session? Because commands may store arbitrary variables and functions in the environment. For example, it would be common to run rvm, which is a bash function added by a previously run command.

Why can't I use && or ;? Because I need the stdout and stderr or each command individually, and this would mix them.

Why can't I use screen? Because we want to run commands automatically, not as part of an interactive session. Consider the difficulty of this: I'd have to run the command with input form redirection, output via redirection, and then scp the results back. I'd need to echo $? to get the exit code. I wouldn't be able to timestamp the stderr/stdout. I wouldn't be able to timeout the command. And it would be incredibly buggy.

  • 1
    Please edit your question to add your script. We can then find why env && screen and ; won't work for your script. Commented Dec 19, 2011 at 3:53
  • @Jeremy: there is no script. This is a program to run commands remotely under various configurations and store their results. Commented Dec 19, 2011 at 3:55

4 Answers 4


If you need something to be able to capture output, interact with a shell session and/or interact with pty applications like screen you could try writing in a higher-level language's SSH library, like Ruby's Net:SSH. It has a bit of a learning curve, but would accomplish everything you need, I believe.

  • We are using an ssh library (jcraft.com/jsch), but I dont think that using one is sufficient information to solve the problem. Can you describe more about how we would use a single bash process for all the commands in Net:SSH? Commented Dec 19, 2011 at 4:09
  • I think Kyle has the right idea. Perhaps a higher level language (perl,python,ruby) would satisfy. Each language has syntax for executing Bash scripts and *nix commands. Commented Dec 19, 2011 at 4:09
  • I've used Net::SSH for far simpler things that you're suggesting (i.e. not stringing commands together) but it appears from the API docs that you can create a "channel" object and "request a pty" at which point you can interact with the shell directly. Seems to be some work involved in retrieving exit codes, something about parsing data.read_long. Anyway I was just suggesting that something like this would be cleaner than a huge nasty ssh command inside of a bash script that writes out result files and concatenates them together (for this, I am speaking for experience ;]).
    – Kyle Smith
    Commented Dec 19, 2011 at 4:15
  • @Kyle: sure, I agree. I guess I'm just looking for specifics like "ptys are able to do X, channels can do Y, so you just need to Z the X and W the Y, and you should be all set". Commented Dec 19, 2011 at 4:50

I need to access the stdout, stderr and exit code of each command separately. This makes it difficult to chain the commands using && or ;.

Well, I don't know if I really recommend this, but you could write a shell function like this one:

function prefix ()
  while read -r ; do
  echo "$OUT"

function run_cmd ()
  echo "Running command << $@ >> . . ."
    ( ( "$@" ; echo "\$?: $?" >&3 ) | prefix 'STDOUT: ' >&3 ) 2>&1 \
    | prefix 'STDERR: '
  ) 3>&1

Then if ./foo.sh bar prints out foo\nbar\n to standard output and baz\nbip\n to standard error, and exits with code 1, then run_cmd ./foo.sh bar will print something like:

Running command << ./foo.sh bar >> . . .
$?: 0

That should let you chain a sequence of commands, provided you use run_cmd for all of the ones that you need STDOUT/STDERR/exit-codes for. (But since the command itself is run in five-layers-deep of subshells, you can't use run_cmd to run any command that needs to set environment variables for use by other commands. Hopefully you don't have any commands that both give output and set variables?)

  • um, yeah :( Right now, rvm does, but this is for arbitrary commands so I can't live with this problem long term :( Good answer though. Commented Dec 19, 2011 at 4:53

Did you already take a look at pssh? It covers the most of your requirements, and with some creativity and imagination it may well help you:

  • Can send the commands to multiple servers at the same time
  • Is able to output in real time
  • stdout and stderr can be saved to separate files
  • reports you if command X at server Y succeeded or not
  • can receive input from stdin and send that to servers
  • has configurable timeout

And so on...


I also believe Kyle's approach would be the most promising. If that doesn't work out, you could write your main script in a way that it goes in an endless loop where it looks for command files which it then sources and makes stdout, stderr and exit code available in a form usable for your purpose.

You would then insert the command files via an alternate channel, likely just scp  or something similar.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .