4

I have a situation where I'm testing a new filesystem that has an issue with some metadata. All of the files are intact, however I ultimately have to effectively move every single file off to another volume and then immediately back in place.

Obviously mv won't do the trick since it's not able to preserve all attributes (particularly all timestamps). I was thinking something more along the lines of a find command whilst exec'ing cp -p /original/path/to/file /tmp/location/file, rm /original/path/to/file, cp -p /tmp/location/file /original/path/to/file, rm /tmp/location/file. Perhaps all of these commands in a script that is passed to the find exec?

I'm not sure of the most efficient approach here, but would like some quick input in order to make sure I cover all of my bases and have a precise command that will not leave me mourning some kind of data attribute loss when it's all said and done. I have many hundreds of gigabytes to shift back and forth so I need to be both as careful and as efficient as possible.

Here's the solution I've come up with so far and would love some input:

Shell script safe_move.sh:

#!/usr/bin/env bash
SRC_FILE_AND_PATH="$1"
SRC_BASENAME=`basename "$1"`
DESTINATION_PATH="/mnt/tmp"

cp -a ${SRC_FILE_AND_PATH} ${DESTINATION_PATH}/${SRC_BASENAME}
rm -f ${SRC_FILE_AND_PATH}
cp -a ${DESTINATION_PATH}/${SRC_BASENAME} ${SRC_FILE_AND_PATH}
rm -f ${DESTINATION_PATH}/${SRC_BASENAME}

And this will be called with:

find /path/to/move -type f -exec safe_move.sh {} \;

Any thoughts or corrections?

  • I did find a problem where this will ALTER folder dates due to modification of the folder's children... hrm. Not sure how to overcome this one yet.
1
  • cp -a with find is appearing more and more to be the likely answer in this scenario, however I have seen some valid points that perhaps tar may handle some scenarios as good or better than cp.
    – ylluminate
    Apr 28, 2012 at 6:20

3 Answers 3

1

Consider using tar with -p (--preserve). That way you also have a nice tarball to keep around... you know... just in case. There is no metadata that I am aware of that would be left behind. I also disclaim any responsibility if I am found to be wrong. =)

EDIT: symlink dates won't be preserved! See?! Never trust a cat with root privileges.

Other options include cplv but you don't have a volume group that is at least the size of the source. cpdup too. If it's all on the same volume group, rsync might be an option.

5
  • The main problem I have there is that I can't really preserve all the files. The only volumes I have to "move" these files onto and back off of is a very small fraction of the size of the extremely large volume set I'm working with. Ultimately it's going to be best to do an and off again, but sure, tar could be used as well.
    – ylluminate
    Apr 28, 2012 at 6:00
  • @ylluminate Updated my answer with some info. If you don't have enough space to do this comfortably, make sure you sit everyone down that is involved, tell them the situation and get written acknowledgement of their approval. It's miserable working with less than what you need to accomplish a task, but it's even more miserable when you get blamed, booted and blacklisted. =|
    – Wesley
    Apr 28, 2012 at 6:02
  • It's me, myself and I. My server and my testing of the fs, so I am just attempting to get the most elegant solution in place possible. Ultimately I have to shuffle about 2.5TB of data off to a 100GB space and back on. Again, I figured I'd like to chat with some like minded folks before pulling the trigger on my solution since it IS my data... ;)
    – ylluminate
    Apr 28, 2012 at 6:05
  • Looking at the solution I came up with above, you can see that I found an issue in my approach with it altering my subfolder date. Do you have a specific use example with tar that you can give in replacing cp in there whilst retaining folder attributes?
    – ylluminate
    Apr 28, 2012 at 7:04
  • @ylluminate Alas, I do not. Perhaps the folks at U&L.stackexchange can help? We'll see if it can get voted for a migration over there.
    – Wesley
    Apr 28, 2012 at 21:00
1

I noticed that you've marked this with the FreeBSD tag. If FreeBSD is your OS, then

dump -0 -f /path/to/destination/file /what/to/dump

The other commands you will need are

newfs /dev/ada0p1
mount /dev/ada0p1 /mnt
cd /mnt
restore -r -f /path/to/destination/file

You want to replace these device nodes with ones that are appropriate for your system. Also, you want to stick to dumping whole partitions and shy away from dumping parts of a partition.

0

I found that rsync ultimately does the trick very well, except you have to remove the leftover folders that the process leaves behind. Unfortunate, but since the folders consume very little space, it's negligible and the rm is acceptable:

rsync -axvES --remove-source-files src_folder /dst_folder/ 
rsync -axvES --remove-source-files /old_dst_folder ./

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .