Cost effective and frugal SAN solution for under $5000?

Question

I will have 3-4 servers that will need SAN access for both IOPS performance, and raw storage capacity. The limit is, cost. Solutions I have seen run $10,000+ for anything 9TB or higher. If I were to build my own, what hardware would you recommend and what software should be used if it is a $5000-7000 spending limit?

It will all be linux environments.

Answer 1

Openfiler can provide the iSCSI infrastructure for the SAN. Your cost then is simply the disks and enclosure.

You might look at something like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021

which provides 20 hot-swap drive bays for SAS drives. Slap in a motherboard of your choosing and some good SAS controllers. Let openfiler handle the RAID-6 and the iSCSI for you. I think you could do it quite cost effectively.

Answer 2

Don't have any hardware recommendations off the top of my head, but as for software and networking you should definitely go for iSCSI. It will allow you to run the SAN network over your existing ethernet infrastructure, and adding more clients is basically free if you use the open-iscsi initiator.

Of course, having a dedicated switch or at least a VLAN is recommended, as well as a dedicated port on each machine for iSCSI traffic. You can get by without this but performance will suffer.

For the target, you can run OpenFiler, which will basically give you an appliance-like solution and ease the burden of configuration.

Answer 3

To really get an answer to your question you'll need to give the community a little more information. Not only the physical specs that you require but also what features are required.

Physical Specs
Interconnect type (Ethernet, FC, etc and what speed and how many ports)
Capacity Required
Form Factor (does it need to be rack mount or stand alone)

Features
What sort of redundancy do you need? Power supplies, controllers, Multi-Path IO
Do you need snapshots?
Do you need replication?
How are you planning on backing it up?

With that said even the entry level Enterprise single controller iSCSI SAN solutions you're going to be looking at $10k+ for 2-5TB of storage. That's just the where the market is. If you're running 4 servers off this one SAN just remember that is a single point of failure that could take down all the services that those servers are running. So you need to plan accordingly.

My favorite storage platform for do it yourself these days is OpenSolaris. With ZFS you get a lot of features only found on the commercial SAN solutions. There's also Nexenta which is commercially supported storage platform based off OpenSolaris and ZFS. It may be a good compromise.

Answer 4

EDIT: After another year or two of experience with the Promise, I can safely say - AVOID THIS PRODUCT! Under any amount of heavy load the Vtrak would timeout on certain iSCSI commands forcing us to limit its usage. The last nail in the coffin was when the Vtrak failed 1/2 of the drives after a reboot resulting in data loss.

We put a Promise Vtrak iSCSI SAN in place a couple years ago for some cheap storage and it has worked great for us. You can use off the shelf SATA drives (I recommend Seagate Barracuda ES2) which cuts down on your cost. It will support up to 16 1TB drives depending on the model.

Don't forget to include a dedicated switch (or two) with jumbo frames support and some dedicated NICs for your server. Two NICs and two switches will give you some added redundancy.

http://promise.com/product/product_detail_eng.asp?segment=undefined&product_id=185

Answer 5

You can pick up some very nice, but older SAN solutions on eBay. A friend of mine was looking at an EMC CX500 with like 70 disks for $11k. I'm sure you can find a smaller unit from EMC or another vendor in your price range.

Answer 6

Many people have been excited about the DroboPro. Take a look. It supports everything you want at the price point you provided.

Answer 7

All-Linux environment? All the servers in one place, or at least attached to a single switch? Don't use iSCSI, Use ATA-over-Ethernet (AoE) instead. Similar concept to iSCSI, just cuts out a networking layer to let you use more of your bandwidth. Spend your money on a bunch of harddrives, a case to put them in, extra networking ports, a decent hardware RAID card and a switch capable of Flow Control (a low-end "web managed" switch should be capable of this).

Alternatively, cut the idea of having a SAN at all and keep storage local to each machine. That way you get the 3GB/s performance of a snazzy fibre-channel SAN for the cost of a motherboard capable of SATA II. Buy some extra network cards ($10 each?) and have a dedicated link between each machine, then mirror volumes over those network links between machines.

Answer 8

I'd recommend looking at a nexenta/pogo combo. Super robust platform that nicely fits your budget.

Answer 9

Well, this is a tough one. You want both high IOPS and high raw storage capacity. Thats kind of the best of both worlds, wich costs money.

I think you have to settle on one of the goals and then try to find a product that suits.

Personally I can recommend Nexsan products. They have rubbish names for their products, their manuals and GUI look like it was made in 1995 but their products are EXTREMELY stable.

We have one 42TB SATABeast with dual controllers (FC and iSCSI), zero problems so far (1 year old). We also have a ATABoy, zero problems except disk dying of age.

They just released their SAS storage arrays. Perhaps a SASBoy will fit your needs? IIRC, you can mix SAS and SATA in the same case (high I/O on SAS, high capacity on SATA).

Answer 10

I am a big fan of the NetGear products. Although they are SATA-2 drives they provide pretty cheap performance. The newly-released 3200 offers up to 12 SATA drives and their proprietary XRAID-2 is pretty fast and convenient.

http://www.readynas.com/?p=2251