14

I am using rsync(1) to keep a local debian repository updated. Recently, the disk I am using to store it started running low on space, so I decided to use symlinks to allow me to move some of the directories to another similar sized disk.

Unfortunately, it seems that rsync is deleting the symlinks and refilling the nearly full disk. After some searching, I discovered the --keep-dirlinks option to rsync, which seems tailor-made to fix my problem.

Only it doesn't. The symlinks on the target are still getting deleted.
Here is my rsync command:

rsync --recursive --keep-dirlinks --links --hard-links --times --verbose \
 --delete --delete-excluded $EXCLUDE $SOURCE_EXCLUDE \
 $RSYNC_HOST::$RSYNC_DIR/pool/ $TO/pool/

The EXCLUDEs expand out to eliminate a large number of architectures that I'm not interested in, along the lines of --exclude binary-alpha/ --exclude disks-alpha .... Before the rsync launches, my pool directory looks like this:

lrwxrwxrwx  1 root root   23 2014-09-22 13:58 contrib -> /u2/debian/pool/contrib
drwxrwxr-x 62 root root 4096 2012-04-09 03:02 main
lrwxrwxrwx  1 root root   24 2014-09-22 13:58 non-free -> /u2/debian/pool/non-free

Once the rsync kicks in, I get this:

receiving incremental file list
deleting non-free
deleting contrib
./
contrib/
contrib/a/

And so on, and the symlinks are replaced with directories full of files.

Are some of the other rsync options interfering with --keep-dirlinks? Which ones can I not combine? Or is it the order of options that's causing my problem?

Improve this question

1 Answer 1

Reset to default
8

The --delete and --delete-excluded options interfere with your plan, as they notice that the symlink is not there on the source and thus delete it.

This only happens when you tell rsync to copy the whole directory with the source/ target notation (the trailing slash after the source). If you use source/* target instead, the shell will expand the list of files and directories to copy and this will not happen (but try with --dry-run first).

However, instead of this fragile setup I would recommend to combine the two disks with the help of LVM to form one large volume not needing this kind of trickery.

Improve this answer
3
  • Thanks! I've changed the script and it seems to be working. This will give me a little time -- the disk is still going to tend to fill up, but I have time to shift things. You're right about combining the two with LVM. I'd already been considering it, but dragging my feet since it means moving things off and then moving them back on. Better sooner than later, though, I guess.
    – unkilbeeg
    Commented Sep 30, 2014 at 15:34
  • I had a similar problem that was resolved by using --keep-dirlinks without the -a option I was using. "Minor" annoyance as for the third paragraph, the file systems I am using are 64 T byte SAN "disks." While it is possible that multiples could be combined into yet bigger, say 256 terabyte drives, I expect the admin people have reasons for the sizes we have. Sometimes what looks like "trickery" is just working out how to live with what you have.
    – user1683793
    Commented Oct 5, 2018 at 14:29
  • My ultimate solution was new hardware. The original machine was old enough to only support IDE drives, and the biggest drive you could get in those days was 512G.
    – unkilbeeg
    Commented May 8, 2019 at 16:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .