Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a backup routine that previously ran from one disk to another on the same server, but have recently moved the source data to a remote server and are trying to replicate the job via scp.

We need to run the script on the target server, and we've set up key-based scp (no username/password required) between the two servers. Using scp to copy specific files and directories works perfectly:

scp -r -p -B buser@10.0.0.1:/mnt/disk1/bsource/filename.txt /mnt/disk2/btarget/

However, our previous routine iterates through directories on the source disk to determine which files to copy, then runs them individually through gpg encryption. Is there any way to do this only by using scp?

Again, this script needs to run from the target server, and the user the job runs under only has scp (no ssh) access to the target system.

The old job would look something like this:

#Change to source dir
cd /mnt/disk1

#Create variable to store 
# directories named by date YYYYMMDD
j="20000101/"

#Iterate though directories in the current dir
# to get the most recent folder name
for i in $(ls -d */);
do
 if [ "$j" \< "$i" ]; then
  j=${i%/*}
 fi
done

#Encrypt individual files from $j to target directory
cd ./${j%%}/bsource/
for k in $(ls -p | grep -v /$);
do
 sudo /usr/bin/gpg -e -r "Backup Key" --batch --no-tty -o "/mnt/disk2/btarget/$k.gpg" "$/mnt/disk1/$j/bsource/$k"
done

Can anyone suggest how to do this via scp from the target system? Thanks in advance.

EDIT: With all the bash one-liners I see flying around here, I was half-expecting within a few seconds to see some epic thing pop up with cat, grep, and maybe a '>>' or two between a few pipes. Ah, well. ;-)

share|improve this question
1  
This doesn't address your question, but you should pipe find into while (with type d or -type f) instead of using ls (mywiki.wooledge.org/ParsingLs). –  Dennis Williamson Mar 16 '10 at 0:01
    
Thanks, that's a good tip. –  nedm Mar 16 '10 at 0:08
add comment

3 Answers

up vote 1 down vote accepted

Run a similar version of the earlier script on the source server using cron, where it archives and encrypts all the files to a directory. At an appropriate interval following that, have a cron on the target server rsync over SSH that location. You could scp but rsync has many advantages.

Otherwise, I'd scrap your requirements and run an on-demand hybrid solution from the target using SSH pipes.

share|improve this answer
    
Thanks, I suspect this is what we'll have to do. –  nedm Mar 15 '10 at 22:29
    
See my answer below, though I think we'll still have to run the sorting part on the source server so I'm giving you the answer points. Out of curiosity, if one isn't updating existing files, what's the advantage of rsync over scp? –  nedm Mar 16 '10 at 0:02
add comment

I don't know, if sshfs could be a solution for you. This way you wouldn't have to allow unrestricted ssh, and you wouldn't even have to change your bash script. (I guess the performance would be somewhat worse though, and maybe you have additional requirements that won't allow this approach.)

share|improve this answer
add comment

UPDATE: After some experimentation, I've worked out how to pipe scp to gpg from the target server via stdout:

scp -r -p -B buser@10.0.0.1:/mnt/disk1/bsource/filename /dev/stdout | gpg -e -r "Backup Key" --batch --no-tty -o "filename.gpg" 

I think for the other half of this I'll run the sorting job on the source server and output the resulting directory name to a text file in a static location, which will be scp'd first from the target system, then cat'd into the variable for the source location.

share|improve this answer
    
Ah, neat. I usually use ssh as opposed to scp for similar tasks. Glad you found a viable solution. –  Warner Mar 16 '10 at 2:45
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.