0

First I thought that rsync isn't syncing everything described here Limitations of rsync? Not syncing everything

Now after a day I found out that it was actually still working/ running and now all the files are there. The problem is: My zip is instead of 1gb only 200mb.

Because the zip was created I thought rsync is "finished" and just doesn't download everything. My logic was: tar only gets executed when rsync is finished. But as it seems that is not the case.

How can I make tar wait until rsync is completely finished? So I don't end up with archive that's not complete?

This is my code:

USER="myusername"
SERVER="myserverip"
PORT="22"
SOURCE="shop"
TARGET="/volume1/BACKUPS/MyFolder/MySubfolder/data/"
LOG="/volume1/BACKUPS/MyFolder/MySubfolder/backup_data.log"

mkdir /volume1/BACKUPS/MyFolder/MySubfolder/

rsync -avz --progress -e "ssh -p $PORT" $USER@$SERVER:$SOURCE $TARGET --delete >> $LOG 2>&1
tar -zcvf /volume1/BACKUPS/MyFolder/archivename-$(date +%Y-%m-%d_%H_%M_%S).tar.gz -C /volume1/BACKUPS/MyFolder/MySubfolder/ .

This is maybe a second question, but somehow related to "only do something when the first thing is finished". How can I detect that the archive is also "finished"? So I don't end up opening/ unpacking an archive that still getting processed? Something like rename the file after the archiving is actually finished. How can I make sure that this renaming doesn't just start at some time and wait until the line before is actually ready, like with rsync?

0

I don't see why this should run async, as long as there is no ampersand & in your original script at the end of line. Are you running this as a cron job? If yes, is the execution frequency maybe too high?

Logs from rsync should actually help here, at least to see if it is really finishing. Did you check them?

Apart from that, there are several ways to make these executions more controlled, as in 'do this if last was successfull'. The most simple one would be

rsync -abc --exwhyzed xxx && tar -abc xxx

note the double ampersand above. The latter command is then only executed if the first one has completed successful.

A more descriptive way would be if statements. Like

if rsync -abc --exwhyzed xxx ; then
    tar -abc xxx
fi

Also, why do you use the --progress option in combination with the verbose flag? When running nun-interactively and logging to a file, this doesn't seem very helpful to me. This is just a curious note, however.

3
  • Thanks. I will look into this. Yes, it is a cronjob. But this happened on initial first run and since then I didn't start it again (or the cronjob). The log is pretty spammed because of all the status percentages. But there is nothing in the log that says it's finished. Only for the single files. And there is nothing about any error. What does xxx stand for? Or is this a placeholder?
    – jona
    Jun 26 '21 at 15:51
  • Yes, these are just placeholders, just like the dummy options. It's just about the structure.
    – loopend
    Jun 26 '21 at 15:53
  • These are the last lines of the log 0 100% 0.00kB/s 0:00:00 (total: 58%) (xfer#47349, to-check=1004/85942) sent 1165678 bytes received 200646451 bytes 2028262.60 bytes/sec total size is 792975206 speedup is 3.93 So total was 58% and today (with no change in the log or me starting it again) it is finished (loaded the full thing via ftp and compares the files). And yes, --progress is pretty stupid, I will remove that to make the log more clear and run a new initial run
    – jona
    Jun 26 '21 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.