I have two directories that needs to be merged together. Files in these two directories are all large files (>= 500MB).
What I want to archive: For files in source directory: if it doesn't exist in destination directory,
mv it to the destination directory (which is fast since we are basically creating a new hard link and unlink the source file); if it exist in destination directory, copy the source file there and remove source file.
The most common way to merge directories in Linux system is to use
--remove-source-files option. But this is slow because it will do copy operation even the destination file doesn't exist.
Any better ideas? Thank you.