Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a script, written in bash, which runs from cron. One of the first things it does is SSH to a remote host and retrieve a list of files in a directory. Everything works fine when run from the command line, but not from cron.

The section of the script originally looked like this:

FILES=$($SSH_BINARY -i $SSH_KEY $SSH_USER@$REMOTE_HOST "ls $REMOTE_DIRECTORY")
echo "Got files = $FILES"

I added an echo statement above that line (shown below) to prove it wasn't a path or variable issue:

echo "$SSH_BINARY -i $SSH_KEY $SSH_USER@$REMOTE_HOST \"ls $REMOTE_DIRECTORY\""

If I take the resultant output line and run it as the same user cron will (root), it works without issue.

Thinking it may have something to do with assignment to a variable, I modified the FILES= line to read (thus, putting the output directly into my last_run_output file):

$SSH_BINARY -vv -i $SSH_KEY $SSH_USER@$REMOTE_HOST "ls $REMOTE_DIRECTORY"

The cron entry looks like this:

34 * * * * /root/path/to/my/script/get_logs.sh > /root/path/to/last_run_output 2>&1

So, PATH, variable assignment, and permissions issues aside, I started using debug flags in ssh. I ran once from the command line, then from cron, and compared the outputs. Here are some highlights from the diff:

The - side is the unsuccessful attempt, the + side is the successful attempt, run outside of cron.

@@ -87,9 +77,7 @@
 debug1: Remote: X11 forwarding disabled.
 debug1: Remote: Forced command: /home/sshacs/acssshsink netstorageuser
 debug1: Authentication succeeded (publickey).
-debug2: fd 4 setting O_NONBLOCK
 debug2: fd 5 setting O_NONBLOCK
-debug2: fd 6 setting O_NONBLOCK
 debug1: channel 0: new [client-session]
 debug2: channel 0: send open
 debug1: Entering interactive session.

I can't explain why these extra file descriptors are mentioned in debug2 when run from cron, but it appears to be related (notice read<=0 rfd 4 len 0 line below):

@@ -100,20 +88,672 @@
 debug2: callback done
 debug2: channel 0: open confirm rwindow 0 rmax 32768
 debug2: channel 0: rcvd adjust 131072
-debug2: channel 0: read<=0 rfd 4 len 0
-debug2: channel 0: read failed
-debug2: channel 0: close_read
-debug2: channel 0: input open -> drain
-debug2: channel 0: ibuf empty
-debug2: channel 0: send eof
-debug2: channel 0: input drain -> closed
+  [[ Very large output of the ls command ]]
+debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
 debug2: channel 0: rcvd eof
 debug2: channel 0: output open -> drain
 debug2: channel 0: obuf empty
 debug2: channel 0: close_write
 debug2: channel 0: output drain -> closed
-debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
 debug2: channel 0: rcvd close
+debug2: channel 0: close_read
+debug2: channel 0: input open -> closed
 debug2: channel 0: almost dead
 debug2: channel 0: gc: notify user
 debug2: channel 0: gc: user detached

Any ideas are greatly appreciated.

share|improve this question
    
Is the remote directory the same when run from each? SSH command should be: $SSH_BINARY -vv -i $SSH_KEY $SSH_USER@$REMOTE_HOST ls "$REMOTE_DIRECTORY" –  ewindisch Jan 19 '11 at 17:54
    
ewindisch, the command is exactly the same when run from each, as I use the output of the 'echo' line to determine what to run outside of the script. Also, what does '-w' do? My ssh binary doesn't appear to support it. –  Kyle Smith Jan 19 '11 at 18:01
    
One suggestion: Can you set your shebang to #!/bin/bash -ex (assuming you use bash), this will echo all executed commands to stdout (-x) before running them and exit if an undefined $VAR is referenced in the script (-e). Useful for debugging this stuff. –  SmallClanger Jan 19 '11 at 18:09
    
SmallClanger: Very good idea, unfortunately in this case I found nothing unexpected. Seems to do what I expect.. :( I will keep -x in mind in the future though. –  Kyle Smith Jan 19 '11 at 18:15

2 Answers 2

up vote 5 down vote accepted

Try adding "-t -t" to your ssh connection options. This will; force a psuedo-terminal to be allocated.

share|improve this answer
    
THANK YOU! We've been looking at this for far too long, we had actually tried the -t option but hadn't read to the part where 'multiple -t's will force a psuedo-terminal even if a local psuedo-terminal is absent'. Thanks again! –  Kyle Smith Jan 19 '11 at 19:50

You should debug your script further. Two things to try:

Rwrite it to do file output directly instead of through output redirection. That is, in the script do this:

$SSH_BINARY -i $SSH_KEY $SSH_USER@$REMOTE_HOST "ls $REMOTE_DIRECTORY" > savefile

Run your script with bash -x inside cron and save result. Change first line of script to #!/bin/bash -x and try something like

34 * * * * /root/path/to/my/script/get_logs.sh > script_debug 2>&1

that should help you narrow down exactly what your script is doing. I suspect redirection of output in cron is not working correctly in this case.

Also, dumb question: have you verified that root can write to /root/path/to/last_run_output? Have you verified that you don't have noclobber set somehow when the script runs in cron?

EDIT: further troubleshooting ideas based on comments from OP.

So none of the above ideas seem to be working. What about saving the fielist in a file on the remote machine and scp'ing it back? That would eliminate any weird quoting or input redirection problems.

share|improve this answer
    
Phil, thanks for the ideas. I had actually already tried eliminated the output redirection in the cron entry (thinking it may be related to fd4, or something) and it made no difference. I did also try redirecting at that command instead of in cron, same problem. At SmallClanger's suggestion (see OP comments), I tried a -x as well. Nothing unexpected. Also I have write access to that file, it gets replaced with useful debug at every run. –  Kyle Smith Jan 19 '11 at 18:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.