34

I would like to write a simple backup script that saves some data to a FAT drive. Should I reformat the drive and use a better file system or is it possible to use rsync with FAT? If so, what problems might I run into? Would performance be a lot worse?

EDIT: This is on linux, didn't even know there was a rsync for windows. The sources are various file systems (it's a mess), and the destination is currently formatted with FAT32.

Thank you for your answers, I'll probably go for a reformat, since I'm not completely sure about the file sizes we'll have.

1

7 Answers 7

15

I would recommend reformatting to a Linux file system if you possibly can. As mentioned, FAT has relatively low file size limits and might not handle permissions and ownership quite right. More importantly, FAT doesn't track modification times on files as precisely as, say ext3 (FAT is only precise to within a 2 second window). This leads to particularly nasty behavior with rsync as it will sometimes decide that the original files is newer or older than the backup file by enough that it needs to re-copy the data or at least re-check the hashes. All in all, it makes for very poor performance on backups. If you must stick with FAT, look into rsync's --size-only and --modify-window flags as workarounds.

48

I use rsync to backup my photos I store and process on laptop running Linux (Ubuntu 10.4). I backup them to a very basic NAS with 1TB hard disk formatted as FAT32. The NAS case and firmware is very basic, so it doesn't allow to reformat the drive.

The command I use is:

$ rsync --progress --modify-window=1 --update --recursive --times \
  /home/mloskot/Pictures /mnt/nas/Pictures

To allow correct time comparison --modify-window=1 option is used, because FAT32 records file timestamps with 2-seconds resolution which is different to filesystem(s) used on Linux. The --update to avoid unnecessary copying of existing files - it behaves like incremental backup.

In order to do size-based comparison, you can specify --size-only option.

5
  • Thanks for sharing, I will be checking out your combination of flags :-) The rsync manual is a bit bulky...
    – DutchUncle
    Jul 5, 2012 at 14:36
  • 3
    the --modify-window=1 makes a huge difference. It solved the - i'm now done in milliseconds instead of minutes. Nov 13, 2014 at 10:33
  • 4
    For some reason I need to use --modify-window=2. When using 1 it still copies all files. Copying from NTFS to FAT32.
    – Luc
    Dec 7, 2015 at 18:40
  • 1
    Occasionally, even with --modify-window, rsync would insist on copying over all my files. I am using the --size-only option now and that's working ok. That was from Mac OS Extended (Case-sensitive, Journaled) to FAT32. Apr 3, 2018 at 4:02
  • Probably a good idea to add '--ignore-errors --max-size=4GB' to avoid crashing on files that would not fit into FAT32.
    – Maksym
    Jul 8, 2018 at 14:56
3

Is this rsync on linux to/from a FAT (which version of FAT?) disk or are you using the Windows version cwRsync?

Either way FAT16 and 32 work with both versions (haven't tried FAT12 myself). If you're on a linux system I would expect performance from a FAT filesystem to be a little worse than using a linux-format filesystem as in general they are slightly slower (happy for someone to put me right on this one if I'm wrong).

1

Yes reformat! You should always use a better file system than FAT unless you have to for compatibility reasons!

I'd suggest you format the drive using the same file system that you are rsyncing from otherwise you will potentially lose permissions and attributes that are set on your files and directories.

FAT (presumably FAT32) could be problematic to due maximum file sizes (4GB), maximum volume sizes (2Tb - one day anyway!), fragmentation etc.

However if none of this matters and you just want a simple, fast & maximum compatible file system then FAT will be OK. Speed wise, due to the lack of journalling, permissions etc, FAT could possibly out perform the alternatives - NTFS/Ext3/HFS...

1

Is there a reason you're using the FAT drive? Rsync should work to save data, but you'll lose meta information about files...ownership, permissions, etc...and you risk having issues if you ever rsync large files since FAT has a filesize limit of 2 gig.

Performance I can't comment on, I've never run benchmarks on it compared to others. I imagine it also has a lot to do with the drive's own specifications as well as filesystem in regards to that.

If you could, I'd reformat to the native format you're planning to use with the computer (OS X? Linux?), if for nothing else than to keep the metadata and lose the forehead slap in a few months when I end up syncing something that errors out due to filesystem limitations. often sharing with other systems can be done through sharing exports or if you use NTFS you can keep some metadata but still get most cross-system compatibility since the Mac, NT, and Linux all support NTFS R/W ability now.

1

I found that rsync -rtcvP SRC DEST works very well with FAT32.
Instead of using the popular archive (-a) transfer mode this one uses the recursive + times + checksums flags.

The checksums (-c) flag is the essential bit. Comparison by checksums is more robust than comparison by timestamp and file size. Settings this flag avoids transferring existing files to the destination. The transfer will be a bit more CPU intensive, but depending on the contents this might be negligible.
The times (-t) flag preserves modification times on the destination.

0

These are my preferred rsync commands to/from FAT and exFAT filesystems:

SOURCE_PATH="path/to/source/dir"
DESTINATION_PATH="path/to/destination/dir"

# Option 1. **Copy** from source to destination.
#
# dry run
time sudo rsync -hvrltD --modify-window=1 --stats --info=progress2 \
    --dry-run "$SOURCE_PATH/" "$DESTINATION_PATH/"
# actual copy
time sudo rsync -hvrltD --modify-window=1 --stats --info=progress2 \
    "$SOURCE_PATH/" "$DESTINATION_PATH/"

# Option 2. **Mirror** from source to destination. 
# - CAUTION: this deletes files and folders that are on the destination but not
#   the source, thereby making destination dir *exactly match* the source dir.
#
# dry run
time sudo rsync -hvrltD --modify-window=1 --stats --info=progress2 \
    --delete --delete-excluded --dry-run "$SOURCE_PATH/" "$DESTINATION_PATH/"
# actual copy
time sudo rsync -hvrltD --modify-window=1 --stats --info=progress2 \
    --delete --delete-excluded "$SOURCE_PATH/" "$DESTINATION_PATH/"

For details, see my full answer here: Super User: Best rsync settings for copying and mirroring to or from FAT and exFAT filesystems

I booted onto an Ubuntu 22.04 live USB on a 2015 MacBook Air, then used the rsync commands above to back up the whole computer to an external exFAT SSD. To read the internal Apple APFS filesystem, I used linux-apfs-rw. My installation instructions are here.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .