Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I just bought a new 500GB external harddrive. Most of the time I only store operating system ISO's on those things and some movies. Large files anyway, well over 4GiB sometimes, so FAT is out of the question.

So, I'm left wondering. My old external harddrive was NTFS formatted, because it had to be able to connect to Windows boxen sometimes. This new harddrive will not have that requirement. Would NTFS still be the best option? Would one of the ext{2,3,4} filesystems be a good choice for a removable disk? Are there filesystems that I am unaware of that would fit the bill?

"The bill" here would be a stable filesystem that is able to survive an unclean disconnection, is quick, can handle files of over 4GiB and is able to be used on at least kernels 2.6.28 and up.

Update: I hadn't taken xfs into account, and it's large filesystem support is interesting. I'll take a look at the operating systems / kernels I am using (RHEL5, CentOS5, Ubuntu 9.04 and 8.04, Fedora11) to see if those kernels support xfs. I think I remember some of them (the distros) not supporting it by default, but then again, maybe I'm mistaking jfs for xfs here.

The sync option too, is interesting, but that would mean creating UDEV rules for this device on all of my machines, which is not a huge problem, but still something I'll have to keep in mind.

Finally, taken xfs and the sync option, I'm wondering whether the performance gain for xfs and the performance hit with the sync option would be noticeable over the - still rather limited - USB interface.

share|improve this question
    
It's not related to the filesystem, but one suggestion. . . if your drive supports FireWire (400 or 800), use it. It'll perform better than USB. –  Christopher Cashell Jun 11 '09 at 15:04
    
I thought about that, but the FireWire drives (the WD Passport Studio, for example) are on average about 30% more expensive, so I went with the normal version... –  wzzrd Jun 11 '09 at 18:24

7 Answers 7

up vote 7 down vote accepted

Personally I use XFS all over the place (and it sound like that's for personal use).

On an administrational decision it's not that important regarding size, with a 500GB disk you are nowhere near the limits of any filesystem (except FAT as David Schmitt pointed out). Do not consider NTFS, it may have read support but using NTFS under linux is just plain wrong.

I'd still stay with ext3 is is well tested and has the largst toolset available in case someting happens to your data.

share|improve this answer
    
For something that's not critical like carting ISOs around, XFS' interesting corner cases won't hurt, but it's good large file performance will help. –  womble Jun 11 '09 at 8:05
    
exactly why i mentioned my personal use, the OP seems to deal with larger files mostly –  Server Horror Jun 11 '09 at 8:06

The filesystem is almost irrelevant. You want to mount the filesystem "sync" though. That'll cost you write performance, but you can be sure that all the data is written after the cp is finished.

Of course, mount -o remount,async is your friend if you want to temporarily get the performance.

To actually give a specific answer too, I would recommend ext3 with the data=journal and sync mount options.

share|improve this answer
    
I wouldn't say it's irrelevant. . . depending on your usage patterns, you can see a rather significant performance difference. For example, try deleting a lot of very large files on ext3, then try the same action on jfs, xfs, or ext4. –  Christopher Cashell Jun 11 '09 at 15:05
    
Deleting lots of large files doesn't seem to be a problem with wzzrd's disk. –  David Schmitt Jun 14 '09 at 8:38

Most filesystems won't cope very well with being disconnected without being unmounted first, but you can mitigate that by using a journaled filesystem. Ext3 with data=journal would work, as this journals the data, as well as the metadata. It's a little slower, but safer. XJS and JFS are also journaled filesystems, but I don't know if they do journalling of data as well as metadata.

share|improve this answer

I'd go with ext3, hands down. I've heard that ext4 isn't that stable yet (don't shoot me if I'm wrong) it's got journalling, so unclean disconnects shouldn't be a problem.

On ext3, there is always a portion of the disk hidden (5%). This is for when a root drive get's swamped, you can still log in at runlevel 1 to remove some data. On an external hard drive, you can switch this to 1% (to be safe):

# tune2fs -m 1% /dev/sdb1
share|improve this answer
2  
No, you do not want to do that. ext3 reserves that space for two reasons, to reserve space for root, and more importantly, to provide good performance as the drive approaches capacity and fragmentation becomes a bigger problem. It's (almost) never a good idea to reduce or remove the reserved space on an ext3 drive. –  Christopher Cashell Jun 11 '09 at 15:08
    
I can't imagine that it really makes sense to keep a constant percentage of all file systems hidden, no matter how large the file system. While 5% makes sense for smaller file systems, 1% should do just fine for anything that is very large. –  Eddie Jun 12 '09 at 0:28
    
I can follow you on the fragmentation problem, but reserving space for root doesn't seem like a problem for an external data drive. –  Hofa Jun 12 '09 at 7:16

I'd use XFS and carefully remember to unmount the disk before unplugging it.

Edit: Arrf, didn't see the requirement for unclean disconnects. Mount it with "sync" as David Schmitt pointed out. I'd still unmount it before plugging out (when it's possible).

share|improve this answer

If you want something that will tolerate (May not like it) being yanked out is ZFS.

EDIT ZFS is support on Linux with ZFS ON FUSE

share|improve this answer
    
I'm really pretty sure the distros I use do not support ZFS ;-) –  wzzrd Jun 11 '09 at 14:42
    
ZFS is only (mostly) production ready on Solaris. Yes, it runs on BSD, and I think FreeBSD just declared it stable, but I'd definitely give it some time yet. Regardless, it's not supported on Linux, where this was explicitly stated to be needed. –  Christopher Cashell Jun 11 '09 at 15:02
    
And it's my understanding that just unplugging a ZFS filesystem is a great way to make you reformat the drive. Am I incorrect here? –  Matt Simmons Jun 12 '09 at 1:02
    
ZFS is incredibly fault-tolerant, even to the point of checksumming every block. You won't get data corruption without hardware problems (like sudden unmount), and if you do you'll instantly and automatically know it. –  Dan Udey Jun 12 '09 at 7:08

I use ext3 for my external drives for these reasons:

  1. It's a journaling file system, and will thus better survive disconnections
  2. It's the same filesystem I use for / and /usr and /home (etc...)
  3. I know that it supports all of the extra attributes that I need, for example, for selinux, so when I make a backup all attributes are preserved.

If you are using selinux or any other system that makes use of extended attributes or ACLs, make sure that your backup media is formatted using a file system that can back up those very same attributes or ACLs.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.