3

I just bought a new 500GB external harddrive. Most of the time I only store operating system ISO's on those things and some movies. Large files anyway, well over 4GiB sometimes, so FAT is out of the question.

So, I'm left wondering. My old external harddrive was NTFS formatted, because it had to be able to connect to Windows boxen sometimes. This new harddrive will not have that requirement. Would NTFS still be the best option? Would one of the ext{2,3,4} filesystems be a good choice for a removable disk? Are there filesystems that I am unaware of that would fit the bill?

"The bill" here would be a stable filesystem that is able to survive an unclean disconnection, is quick, can handle files of over 4GiB and is able to be used on at least kernels 2.6.28 and up.

Update: I hadn't taken xfs into account, and it's large filesystem support is interesting. I'll take a look at the operating systems / kernels I am using (RHEL5, CentOS5, Ubuntu 9.04 and 8.04, Fedora11) to see if those kernels support xfs. I think I remember some of them (the distros) not supporting it by default, but then again, maybe I'm mistaking jfs for xfs here.

The sync option too, is interesting, but that would mean creating UDEV rules for this device on all of my machines, which is not a huge problem, but still something I'll have to keep in mind.

Finally, taken xfs and the sync option, I'm wondering whether the performance gain for xfs and the performance hit with the sync option would be noticeable over the - still rather limited - USB interface.

3
  • It's not related to the filesystem, but one suggestion. . . if your drive supports FireWire (400 or 800), use it. It'll perform better than USB. Jun 11, 2009 at 15:04
  • I thought about that, but the FireWire drives (the WD Passport Studio, for example) are on average about 30% more expensive, so I went with the normal version...
    – wzzrd
    Jun 11, 2009 at 18:24
  • A deep study about the need of a specific filesystem for removable device as been written by Jonathan Corbet: lwn.net/Articles/512788 Aug 23, 2015 at 15:49

7 Answers 7

7

Personally I use XFS all over the place (and it sound like that's for personal use).

On an administrational decision it's not that important regarding size, with a 500GB disk you are nowhere near the limits of any filesystem (except FAT as David Schmitt pointed out). Do not consider NTFS, it may have read support but using NTFS under linux is just plain wrong.

I'd still stay with ext3 is is well tested and has the largst toolset available in case someting happens to your data.

2
  • For something that's not critical like carting ISOs around, XFS' interesting corner cases won't hurt, but it's good large file performance will help.
    – womble
    Jun 11, 2009 at 8:05
  • exactly why i mentioned my personal use, the OP seems to deal with larger files mostly Jun 11, 2009 at 8:06
6

I'd go with ext3, hands down. I've heard that ext4 isn't that stable yet (don't shoot me if I'm wrong) it's got journalling, so unclean disconnects shouldn't be a problem.

On ext3, there is always a portion of the disk hidden (5%). This is for when a root drive get's swamped, you can still log in at runlevel 1 to remove some data. On an external hard drive, you can switch this to 1% (to be safe):

# tune2fs -m 1% /dev/sdb1
3
  • 2
    No, you do not want to do that. ext3 reserves that space for two reasons, to reserve space for root, and more importantly, to provide good performance as the drive approaches capacity and fragmentation becomes a bigger problem. It's (almost) never a good idea to reduce or remove the reserved space on an ext3 drive. Jun 11, 2009 at 15:08
  • I can't imagine that it really makes sense to keep a constant percentage of all file systems hidden, no matter how large the file system. While 5% makes sense for smaller file systems, 1% should do just fine for anything that is very large.
    – Eddie
    Jun 12, 2009 at 0:28
  • I can follow you on the fragmentation problem, but reserving space for root doesn't seem like a problem for an external data drive.
    – Hofa
    Jun 12, 2009 at 7:16
5

The filesystem is almost irrelevant. You want to mount the filesystem "sync" though. That'll cost you write performance, but you can be sure that all the data is written after the cp is finished.

Of course, mount -o remount,async is your friend if you want to temporarily get the performance.

To actually give a specific answer too, I would recommend ext3 with the data=journal and sync mount options.

2
  • I wouldn't say it's irrelevant. . . depending on your usage patterns, you can see a rather significant performance difference. For example, try deleting a lot of very large files on ext3, then try the same action on jfs, xfs, or ext4. Jun 11, 2009 at 15:05
  • Deleting lots of large files doesn't seem to be a problem with wzzrd's disk. Jun 14, 2009 at 8:38
1

Most filesystems won't cope very well with being disconnected without being unmounted first, but you can mitigate that by using a journaled filesystem. Ext3 with data=journal would work, as this journals the data, as well as the metadata. It's a little slower, but safer. XJS and JFS are also journaled filesystems, but I don't know if they do journalling of data as well as metadata.

0

I'd use XFS and carefully remember to unmount the disk before unplugging it.

Edit: Arrf, didn't see the requirement for unclean disconnects. Mount it with "sync" as David Schmitt pointed out. I'd still unmount it before plugging out (when it's possible).

0

I use ext3 for my external drives for these reasons:

  1. It's a journaling file system, and will thus better survive disconnections
  2. It's the same filesystem I use for / and /usr and /home (etc...)
  3. I know that it supports all of the extra attributes that I need, for example, for selinux, so when I make a backup all attributes are preserved.

If you are using selinux or any other system that makes use of extended attributes or ACLs, make sure that your backup media is formatted using a file system that can back up those very same attributes or ACLs.

-2

If you want something that will tolerate (May not like it) being yanked out is ZFS.

EDIT ZFS is support on Linux with ZFS ON FUSE

4
  • I'm really pretty sure the distros I use do not support ZFS ;-)
    – wzzrd
    Jun 11, 2009 at 14:42
  • ZFS is only (mostly) production ready on Solaris. Yes, it runs on BSD, and I think FreeBSD just declared it stable, but I'd definitely give it some time yet. Regardless, it's not supported on Linux, where this was explicitly stated to be needed. Jun 11, 2009 at 15:02
  • And it's my understanding that just unplugging a ZFS filesystem is a great way to make you reformat the drive. Am I incorrect here? Jun 12, 2009 at 1:02
  • ZFS is incredibly fault-tolerant, even to the point of checksumming every block. You won't get data corruption without hardware problems (like sudden unmount), and if you do you'll instantly and automatically know it.
    – Dan Udey
    Jun 12, 2009 at 7:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .