Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

NOTE: I'm really looking for an SSH-y solution here, more than bash-y one.

I need to execute a series of commands over ssh, programatically. There are many ways to do this, but I can't find any that works with my constraints:

  • some of the commands set variables in the shell. Therefore I dont think I can make this work using multiple shells.

  • some of these variables are functions, not normal values, so I dont think I can simply serialize the output of env to "restore" the old environment.

  • I need to access the stdout, stderr and exit code of each command separately. This makes it difficult to chain the commands using && or ;. I believe this also makes it difficult to use screen, because we cant reliably tell when a command finishes.

  • I need real-time access to the stdout, stderr, etc so we can record when the output occurs. (I use "real-time" to mean "nowish", not "hard real-time" or anything like that).

Update

Why do I need a single bash session? Because commands may store arbitrary variables and functions in the environment. For example, it would be common to run rvm, which is a bash function added by a previously run command.

Why can't I use && or ;? Because I need the stdout and stderr or each command individually, and this would mix them.

Why can't I use screen? Because we want to run commands automatically, not as part of an interactive session. Consider the difficulty of this: I'd have to run the command with input form redirection, output via redirection, and then scp the results back. I'd need to echo $? to get the exit code. I wouldn't be able to timestamp the stderr/stdout. I wouldn't be able to timeout the command. And it would be incredibly buggy.

share|improve this question
1  
Please edit your question to add your script. We can then find why env && screen and ; won't work for your script. –  Jeremy Hajek Dec 19 '11 at 3:53
    
@Jeremy: there is no script. This is a program to run commands remotely under various configurations and store their results. –  Paul Biggar Dec 19 '11 at 3:55
add comment

4 Answers

If you need something to be able to capture output, interact with a shell session and/or interact with pty applications like screen you could try writing in a higher-level language's SSH library, like Ruby's Net:SSH. It has a bit of a learning curve, but would accomplish everything you need, I believe.

share|improve this answer
    
We are using an ssh library (jcraft.com/jsch), but I dont think that using one is sufficient information to solve the problem. Can you describe more about how we would use a single bash process for all the commands in Net:SSH? –  Paul Biggar Dec 19 '11 at 4:09
    
I think Kyle has the right idea. Perhaps a higher level language (perl,python,ruby) would satisfy. Each language has syntax for executing Bash scripts and *nix commands. –  Jeremy Hajek Dec 19 '11 at 4:09
    
I've used Net::SSH for far simpler things that you're suggesting (i.e. not stringing commands together) but it appears from the API docs that you can create a "channel" object and "request a pty" at which point you can interact with the shell directly. Seems to be some work involved in retrieving exit codes, something about parsing data.read_long. Anyway I was just suggesting that something like this would be cleaner than a huge nasty ssh command inside of a bash script that writes out result files and concatenates them together (for this, I am speaking for experience ;]). –  Kyle Smith Dec 19 '11 at 4:15
    
@Kyle: sure, I agree. I guess I'm just looking for specifics like "ptys are able to do X, channels can do Y, so you just need to Z the X and W the Y, and you should be all set". –  Paul Biggar Dec 19 '11 at 4:50
add comment

I need to access the stdout, stderr and exit code of each command separately. This makes it difficult to chain the commands using && or ;.

Well, I don't know if I really recommend this, but you could write a shell function like this one:

function prefix ()
(
  PREFIX="$1"
  OUT=''
  while read -r ; do
    OUT="${OUT}${PREFIX}${REPLY}"$'\n'
  done
  echo "$OUT"
)

function run_cmd ()
{
  echo "Running command << $@ >> . . ."
  (
    ( ( "$@" ; echo "\$?: $?" >&3 ) | prefix 'STDOUT: ' >&3 ) 2>&1 \
    | prefix 'STDERR: '
  ) 3>&1
}

Then if ./foo.sh bar prints out foo\nbar\n to standard output and baz\nbip\n to standard error, and exits with code 1, then run_cmd ./foo.sh bar will print something like:

Running command << ./foo.sh bar >> . . .
$?: 0
STDOUT: foo
STDOUT: bar
STDERR: baz
STDERR: bip

That should let you chain a sequence of commands, provided you use run_cmd for all of the ones that you need STDOUT/STDERR/exit-codes for. (But since the command itself is run in five-layers-deep of subshells, you can't use run_cmd to run any command that needs to set environment variables for use by other commands. Hopefully you don't have any commands that both give output and set variables?)

share|improve this answer
    
um, yeah :( Right now, rvm does, but this is for arbitrary commands so I can't live with this problem long term :( Good answer though. –  Paul Biggar Dec 19 '11 at 4:53
add comment

Did you already take a look at pssh? It covers the most of your requirements, and with some creativity and imagination it may well help you:

  • Can send the commands to multiple servers at the same time
  • Is able to output in real time
  • stdout and stderr can be saved to separate files
  • reports you if command X at server Y succeeded or not
  • can receive input from stdin and send that to servers
  • has configurable timeout

And so on...

share|improve this answer
add comment

I also believe Kyle's approach would be the most promising. If that doesn't work out, you could write your main script in a way that it goes in an endless loop where it looks for command files which it then sources and makes stdout, stderr and exit code available in a form usable for your purpose.

You would then insert the command files via an alternate channel, likely just scp  or something similar.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.