What is the best way to RAID external drives?

Question

I love the fact that you can now buy 1 and 2 TB drives cheap. However that much data on a single drive scares me. What would be the best way to build a raid system out of external drives? I am looking for advice on performance, price, and ease of data recovery should a drive fail. What do I need besides the drives? How many drives should the raid be? Any advice is appreciated. - Thanks.

Answer 1

This kind of depends on what you're trying to keep redundant and what degree of safety you want, and what exactly you're looking for in "safety" (just availability? Redundancy? Backup?)

RAID isn't backup...

And RAID doesn't help beyond drive failure. If you have a controller failure you're going to have an availability issue.

Beyond that there's consideration in that external hard drives often have things like power and heating issues if you're going to use them 24/7. Many work fine, but some do have issues with this, since they're meant to be used for portability and not as a workhorse drive (people used to play with making older iPods into RAIDs and discovered the drives died sooner, also I've read of external drives that would try to power down at inopportune moments and have spin-up issues).

If you're a home user just looking for some RAID setup for some important home data and not customer or business data, I'd look at getting a cheap computer with a couple large drives and turn it into a NAS device with something like FreeNAS. It becomes a network appliance with a web interface for configuring it. Then set it up with mirroring.

Also there are papers out there warning about the dangers of RAID (if you really want heavy-duty stuff) and large disks now, because as drives get bigger it's becoming more likely that you'll hit the point where you will have a drive failure in a cluster and not know it until you're trying to recover from a total drive failure (i.e., you have RAID 5 with three disks, a drive fails. You slap in a new drive to replace drive C. As the volume recovers data from A and B to rebuild C, the system discovers an unreadable spot on drive B. You can't recover the volume because of that bad bit, undetected before, so your volume is hosed if you don't ALSO replace drive B and therefore ruin the RAID volume and then recover from a complete backup; we had this happen to us with a Dell PERC-backed RAID 5. Trust me. It sucks. And with drive sizes increasing to ridiculously huge sizes this is more and more common, leading people to recommend RAID 10 or better if you're running servers that need to be available as much as possible and downplaying using RAID 5). One link that I read before is found here discussing RAID 5 for small businesses (don't use it!).

In other words...playing with RAID at home is fun and can be useful, but overall it won't gain you much unless you have a specific purpose behind it.

Another thing to consider: if you're not using hardware RAID like 3ware's cards, you're going to have more difficulty figuring out which drive is dead unless you have a system in place for knowing that drive A is labeled , so when the RAID system you're using goes ploof you can figure out which physical drive is the issue and swap the RIGHT one. Many hardware controllers for RAID will have blinkies to warn you of drive giving problems. For software raid, it can be a crapshoot unless you have a plan ahead of time.

If you have data that you need to be careful to keep available, I'd look first at backup strategy. RAID is useless there. If you delete your important doc in RAID it's gone immediately. If someone hacks it, it's gone immediately. If something corrupts it, RAID will dutifully duplicate the corruption. RAID is only good for being able to use the system if something fries a drive or if you're looking to play with performance with RAID 0. Backups are what can save your bacon when a system fries or data corrupts.

Oh...and depending on how you configure all this and how your budget runs...adding lots of external hard drives can up your electric bill more than a cheap system with internal hard drives running NAS software to turn it into an appliance :-)

Answer 2

You can buy one of those NAS units, which are purpose built for this sort of stuff, and set it up in RAID6 or RAID10 if you want redundancy, so even if a drive fails you can replace it and rebuild it with relative ease.

This pretty much defeats the purpose of having an external HDD though, as it is hardly considered portable, and if you need to move data around from A->B it is assumed that you will still have this data available on either A or B.

Answer 3

Running RAID on multiple USB and even Firewire drives is not always advisable. I've noticed that on some computers that kinds of configuration randomly drops one drive and then forces rebuild. eSata seems to be more stable, but many eSata connectors are easily to disconnect just by slightly touching the cable.

If you really want to keep the data safe I would recommend buying two drives and backuping the first drive to the second one. You can do this with many tools, even Windows comes with backup utility that actually works quite nicely and does compressed incremental backups. Or you could use rsync, or rsync with GUI, like DeltaCopy.

Best option would be to get the second drive (or third) to a different location, maybe a neighbour (in a different house) or someplace else. It all depends on how important the files are.

Answer 4

I have four WD Elements 1 TB drives in a simple linux md-raid level 5 configuration. These drives are dirt cheap, in fact sometimes cheaper than bulk internal drives.

I imagine you can do much the same with windows dynamic disks.

Unfortunately there is little to be gained performance wise, as multiple drives share the same USB bandwith, but at least I feel a little safer than with single drives.

As for external NAS/DAS units, they are probably fine but defeat the point of cheap external storage.

Answer 5

We backup our data weekly to Tape, but for weekday backups and 'extra' backups we prefer disk. For example we backup our exchange server weekdays in full, and we do all of these to a disk backup system. We needed cheap disk storage, without the expense of server grade equipment. To accomplish this we purchased an

External 8 drive STAT enclosure Found Here

An Adaptec STAT raid card

A mid-range Dell Optiplex tower

Putting the three items together we were able to achieve 8-10 usable TB of cheap disk space, by using 8 1.5TB drives.

To answer your question, how many drives, etc. It depends on how much data you want, but we typically use a RAID 5 array with 4-8 disk. Your mileage may vary depending on what you want to accomplish.

Although probably outside the scope of your project (it was outside of mine), I found .this article interesting. It shows how others are taking advantage of cheap drives. In this example, they built a 67TB system.

Answer 6

The most fault tolerant solution is a RAID 5 or RAID 6 server. To increase speed and fault tolerance, a ZFS RAID-Z OpenSolaris server is better (a discussion from the ZFS developers), although it's not operable on Windows, only OpenSolaris/Free|NetBsd. RAID-Z can handle any RAID mode:

The current RAID-Z algorithm is single-parity, but the RAID-Z concept works for any RAID flavor. A double-parity version is in the works. (http://blogs.oracle.com/bonwick/entry/raid_z)

and is able to take incremental snapshots of the filesystem for remote backup - you wouldn't want the building to burn down and wipe all the RAIDed data out, would you?

performance

A new INTEL X25 M (no affiliate linking) is pretty fast, however if you're just looking for storage any modern disk should do, especially in the capacity bracket you mention.

price

In the size bracket you mention you're quite restricted, however you'll always do better buying individual components than a bundle (bundled NAS packages have horrendously low power/speed processors that often struggle to compute to XOR required for RAID 5/6 at an appropriate speed for the disks).

ease of data recovery should a drive fail

RAID 5 is redundant against a single drive failure, RAID 6 against two. RAID-Z can also handle data corruption (just as likely as physical failure these days).

What do I need besides the drives?

If you don't implement ZFS a RAID controller, otherwise just the disks. And if you are going to build a new box externally the parts for that ;) (DIY: Home NAS Box with OpenSolaris and ZFS)

How many drives should the raid be?

At least 3 for fault tolerance (the RAID array will still work with only 2 drives, but you must replace the failure ASAP), up to as many as your controller can handle. The more the merrier - don't forget that usable yield for RAID 5 is

 min drive capacity X ( # drives - 1 )

and you should always get drives of the same capacity for this reason - the RAID stripes must all fit on the smallest drive.

If you're hooking up to an external NAS, try and use a high bandwidth interface such as eSATA.

Useful links:

A thorough review of decent NAS units

All things ZFS

An old /. post on ZFS vs high end NAS/SAN

Answer 7

Vmware physical disc connect is also possible Extercal disc connect phisical disc vmware Or External disc to vhd clone sector by sector vhd connect vm pc vm pc vhs see phisical disc