Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am tired of worrying about data loss at home. My wife is a semi pro photographer, and essentially all of our family memories are digital (and we ought to convert the ones that are not). I am planning on setting up two systems to host the disks, and run some sort of cluster file system (and then back that up to external drives, which rotate off site weekly) via iSCSI on the disks in the two systems. I run Linux, and need to support Apple OS X, and a little Windows (Windows can be ignored, as it will be used for either VMware experimentation with its own LUNs, or it can live with SMB). So it would be great if there was a Mac native client, but I can live with NFS.

The environment will also have a Mac laptop, a Linux laptop, and a large desktop, which will come and go as far as being attached to the main file system, but they will only be users, they will not contribute storage.

Hardware wise, I am fairly open, low power would be good, as these will likely be on 24/7 (unless I set up my Linksys router to send a wake on LAN packet when someone looks for them?), looking for probably a dual core Atom with an nVidia chipset, so I can have 4GB of RAM, with 2-4 SATA ports, and dual gigabit Ethernet. I plan on buying 5 disks to start (gotta figure one will die, so getting the spare, which will be tested of course, first), and it would be good to have a port (and drive slot in the case) or two free. I will have a desktop and two reasonably beefy laptops for playing with VMs and other intensive work, so I don't think a desktop CPU makes sense (tempting though). Being able to boot from a CF card would be nice.

Rather not deal with proprietary drivers, but as these will be headless (probably), it will not be an issue. I have had issues with ATI drivers causing issues with user switching and powersave/suspend (do you want Planet Penguin Racer to fly, or to not have to log out to let you wife or kids use the computer?).

I would like to reuse cases and power supplies I already have, but they are ATX (old, but not so old as to be AT), will there be issues with small MB (cases I worry about, I expect the MB will take a standard 24 pin ATX plug)?

Hardware I am considering:
ZOTAC IONITX-F-E Atom N330 1.6GHz Dual-Core Mini ITX Intel Motherboard - has 16x pci express
ZOTAC IONITX-A-U Atom N330 1.6GHz Dual-Core Mini ITX Intel Motherboard - comes with powersupply
Asus AT3N7A-I Intel Atom 330/Nvidia ION/ Mini ITX Motherboard - has a PCI slot

All have just one gigabit Ethernet port. I'd prefer the one with its own PS, the Zotac's at least have WiFi (gives some redundancy, with horrible performance).

Summary: FS that might work: Gluster seems good (anyone used it personally? How about laptops that come and go?), pNFS might work too, anything else THAT WILL JUST WORK ONCE IT IS SET UP? (to similar questions, too many people posting links to stuff that is a long way from production ready, or that doesn't have any kind of fault tolerance.)

No: Luster really wants a BIG install. xtreemfs - no redundancy, pvfs - I just don't like stuff that sits on your existing fs, GFS2 and OCFS2 - a pain from what people say here.

cool, but not there - POHMELFS, CEPH, CRFS,

share|improve this question
How much data do you need to store, and how fast will it grow? – Nic Nov 11 '09 at 6:33
Well, hundreds of GB now, my wife can fill a 4GB CF card in a busy day of shooting, and there are things like old photo CDs to import as well, so 1TB should be enough for now, and with an extra port, I can expand easily. I expect that 3 disks will be enough, it is just a question of how often I need to buy larger replacements. – Ronald Pottol Nov 12 '09 at 20:12
Amazon S3 might be a good solution if you have the bandwidth to upload the images. – user21129 Oct 6 '10 at 23:41
I bought a pair of Intel SS4200-e's, 2GB RAM, and 4GB DOMs (flash ide boot disks), though I have not bought the disks yet (been busy). Ceph looks more usable at this point and leverages Btrfs, and Gluster is less than ideal. When I get Ubuntu Server on the DOM, I'll pick a cluster file system. – Ronald Pottol Oct 7 '10 at 19:31

As Nic said, how much data do you need?

For myself, every file I've ever created (going back 20 years, although the old old ones are really small), most of my wife's files (most because she's switched PCs a few times and there are a few odds and ends on older hard drives), every picture we've taken or scanned, and most of our family videos (not the raw files straight from the camera, but lots of "the best" clips) all fit on a 320GB hard disk. It'll take at least 3 years before I get to 500GB.

So I just have 4 copies of everything on external hard drives. One drive stays at home, one small one goes with me, one stays at an office, the last one moves around. I use SecondCopy to back up the files from my laptop to the external drives and I do a backup to at least one of the external drives any time I change more than a couple files. Hard drives are cheap enough that when I need bigger ones, it won't cost much, and I'll put aside at least a couple of the 320s to keep forever.

I'm seriously considering (since my laptop hard drive is almost full) getting rid of the "master" copy of all my stuff that's on my laptop and just treating two of the hard drives as my carry-around storage, and the other two as stationary backups.

share|improve this answer
Just a reminder, harddrives don't last forever. As you move to newer drives, make sure you're moving your old data forward as well. – Nic Nov 11 '09 at 7:23
Yeah, for sure, but I'm used to that, I've been done it moving the data from 3.5" floppies to Zip disks to CDs to DVDs and most recently to hard drives. – Ward Nov 11 '09 at 7:37

Because this is for a home environment you might want to look at getting a Drobo FS. They would support everything you want to do and pro photographers swear by them. And if you have the money you can purchase a second Drobo FS and they will automatically mirror (I have never set this up but I have heard of people who have).

They also support Time Machine which is a big plus.

share|improve this answer

Well you could set up Nexenta with RAID Z2, this will not protect your data against a burglary or fire though. I would suggest taking a look at photo sharing services like Smugmug this will backup and share the photos. Distributed home disk service Wuala you can trade local storage for online storage. The best multi-os backup service I've found out there is CrashPlan they also have a family package.

share|improve this answer
My reasons for wanting a cluster file system is so that if something breaks, backups keep working normally, and the files stay available, as I expect it will be days to a week or two before I can make a non emergency fix. – Ronald Pottol Oct 7 '10 at 19:28

This seems needlessly over complex and hard to manage for a home solution, why not just pickup a second hand baby NetApp with dual controllers from eBay or an auction? Loads of resilience plus almost infinite snapshotting.

share|improve this answer
what about the drives for NetApp? doesn't it require custom firmware HDDs or SCASI/SAS HDDs? – Hubert Kario Oct 7 '10 at 12:17
Yeah, you need netapp provided drives but if you picked up an old FAS2020 or similar they come already filled with disks that might well do the job – Chopper3 Oct 7 '10 at 12:19
They will undoubtedly have high power on hours -- in other words much more likely to fail -- and new NetApp drives are expensive to say the least. If you need just high capacity and moderate IOPS go the google way: cheap hardware but redundant. – Hubert Kario Oct 7 '10 at 13:26

You may be interested in the Backblaze Pod if you have a big storage requirements, or a similar solution (RAID cards + multiplier backplates in custom case)

As for filesystem: have you looked at OpenAFS? The one nice feature it has is disconnected operation.

share|improve this answer
Backblaze pods are still bloody expensive, noisy and probably run hot too. Not ideal for home-office usage! – Tom O'Connor Oct 7 '10 at 13:05
In full (67TB) configuration, they may be a bit noisy (but they use 3000rpm 12cm fans -- they are quiet). Expensive? Hell no, 70% of the cost are the drives them self. Besides, you don't have to use exactly such setup, but using SATA expanders is by far the cheapest solution. – Hubert Kario Oct 7 '10 at 13:20
I had not thought about how you can have AFS replicated (thus getting the fault tolerant aspect), I always though the disconnected operation was very cool, I'll take a deeper look. – Ronald Pottol Oct 7 '10 at 19:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.