8

my group is in the market for plentiful storage. We traditionally used big-iron FC-attached SANs, but they are way expensive and provide much more performance than we require.

We want highly available shared storage, and if it performs like local desktop-class drives, that's fine.

Are there shared disk solutions out there that can get us slower but bigger disks?

We like NexSan SataBeast but they don't have a fully global presence which makes support awkward (we're a global company).

Is there any storage of this type that you would recommend? We'd prefer to use FC-attached storage but we're open to suggestions.

Thanks,

Wout.

EDIT: To answer the requests for clarification, we'd like

  • globally available hardware support
  • tens of TB with the ability to grow
  • Highly Available: Ability to provide service through common failures, so:
    • hotpluggable drives
    • multipath IO
    • active/active controller
    • Anything like this that has a low cost/benefit ratio
  • NFS only might be an option depending on HA-ness and performance
  • Price: whatever it costs, but as little as possible

Basically I'm wondering what the 3D graph of price-performance-availability looks like currently. If there are sharp exponential curves, I'd be interested in the systems in those elbows.

As noted below, this could be done by a farm of PC-level hardware serving iSCSI and host-level software RAID6. This proves that whatever an EMC top-end storage array costs, it is probably too much.

8
  • Could you elaborate on the requirements a bit - roughly how much disk do you need and what is the application. Would a NAS type architecture work or do you need block level storage with a clustered file system? For example, if it is a database, you can use some DBMS platforms with NFS shares (NFS provides adequate locking semantics for this but SMB/CIFS does not). In other cases (e.g. SQL Server) you would need block level storage. If it's just a shared pool of files, you may be better off with a NAS anyway. Jun 11, 2009 at 13:30
  • What is cheap for you? What is "plentiful" storage for you? What is "highly available" for you? RAID? clustered NAS? ...?
    – Toto
    Jun 16, 2009 at 16:05
  • we run a IBM XIV attached to a FC SAN and I really like it. 1 maintenance visit from the IBM technician 1 month after they delivered it to replace 2 faulty disks, no production impact. Running flawlessly for 17 months since. Feb 10, 2012 at 11:25
  • :-O Too localized? Is the group wanting cheap but highly available storage that small?
    – w00t
    Feb 13, 2012 at 11:31
  • @FlorenzKley, hardly cheap but interesting, thanks :)
    – w00t
    Feb 13, 2012 at 11:34

16 Answers 16

8

Depends on your definition of "Cheap". But you could look at HP LeftHand Starter SANs. It's an all iSCSI solution but provides better performance the more you scale it, uses standard servers for the hardware, has solutions in SATA and SAS varieties to fit your budget and performance needs, and is supported by a global, good customer service, company. The boxes can be setup with no redundancy for maximum disk space (and minimal cost), or up to several layers of redundancy so your data basically cannot be lost short of an act of God.

The performance is comparable to FC, and even can even surpass FC the more you scale it.

2
  • The prices aren't on there, and I don't really feel like calling an HP salesman. How much do they run, roughly? Jun 17, 2009 at 2:46
  • 1
    Fully loaded SATA dual server setup (about 4TB fully redundant) would run you about 35k or so.
    – skraggy
    Jun 18, 2009 at 14:40
11

How much storage do you need? If you want something in the 10's to 100's of Terabytes you could look at the Sun X4540 (Thumper). Solaris comes with an iSCSI target if you need block level storage or it can be used as a NAS with NFS or Samba.

For those not familiar with Sun's Thumper, it's a 4U 2-socket Opteron box with 48 SATA disks. You can have up to 48TB of disk on a single machine, although with RAID-5 volumes this would be somewhat smaller.

This blog posting discusses some of the ins and outs of using a Thumper as an iSCSI target.

Another option is the secondhand market. You can get re-certified SANs quite cheaply and many (even older ones such as Clariion CXs) have the option of slower SATA or FC-SATA disks. If you don't mind the power bills you can get second hand Symmetrix gear quite cheaply and that has no issues around licensing for the management software - you can operate it through the console on the controller.

3
  • 1
    seconded! they're a great all-round box
    – Chopper3
    Jun 11, 2009 at 13:09
  • 1
    The Thumper is some very nice hardware, but it's still on the expensive side and it doesn't allow two controllers (well, that would be 2 mainboards). I'm still a fan though...
    – w00t
    Jun 16, 2009 at 19:19
  • NetApp seemed to do OK without redundant controllers for a while. I'm not sure if Solaris supports multipathing for its iScsi target. Jul 2, 2009 at 9:06
4

Ahh, haven't got enough reputation points to leave comments yet, so it's a seperate post I'm afraid.

With regard to the HP Lefthand pricing, a good site to grab the pricing on the Lefthand kit without calling HP is the StorageMojo, the StorageMojo also has list prices for most other manufacturers.

You're looking at about £40k for a 2 node setup with a total of 24 x 450GB SAS disks (10TB raw capacity), including the basic hardware support for 3 years. Slap on another £6k if you want the Premium support. This gives you pretty good redundancy and all the software features like Thin Provisioning, SnapShots, Remote Copy, Multi site DR which normally cost extra. The nodes hardware are based on the HP ProLiant DL185 G5 server (so that's 12 disks per node).

It came in at half the cost of an equivalent NetApp solution.

I think with HP having bought them out, the LeftHand solution now offers pretty good bang for the buck with some proper Enterprise grade support behind it, important when things go pear shaped and you need assistance in the wee hours.

1
  • Thanks for the explanation and the link! One question, when I look up SATA prices for Lefthand, I get P-2060-1TB07-HD: Spare NSM 2060 1.0TB SATA Drive, $2,380. Is this for 1 drive? Seriously? 1TB SATA drives normally go for 99$...
    – w00t
    Jun 17, 2009 at 15:09
3

Take a look at the Xiotech Emprise 5000. With their sata "diskpac"s you can get 16TB in 3U with redundant controllers and battery backed cache.

If you're willing to drop FC for iSCSI Dell's MD3000i or Equalogic products are the cheapest "name brand" SAN.

And if you really wanna get your freak on, check out Coraid's AoE. Though I'm not sure to what degree they meet your global presence requirements.

2
  • +1 for "Get your Freak on" and AoE. Jun 11, 2009 at 13:24
  • +1 again for CORAID and AoE
    – rthomson
    May 25, 2011 at 16:13
3

We set this up for less than $AU3000 (so, probably halve that for the US market):

  • Dual Core 2.4Ghz
  • 2x Adaptec SATA RAID cards
  • 4x Gigabit NICs
  • 4U Rackmount Case
  • SATA Hotswap backplanes for case (total of 10x trays)
  • Loaded her up with off-the-shelf Seagate SATAs (with 5 year warranties, but see this SF question if you plan on doing that)
  • Installed FreeNAS and used as an iSCSI target (although you have your choice of connectivity)

Worked like a charm, cheap massive storage with connectivy out the wazoo.

1
  • 1
    This is the fun solution :-) Only problem is supportability, both from a software and hardware standpoint (OS patches, hardware compatibility, subtle driver failures, ...). It's also not highly available but given the low cost it is feasible to have multiple of these and to use host-based software RAID to make it highly available. This question has thought me that there is a cheap, consumer-hardware solution space and an expensive, enterprise solution space, with not much in between.
    – w00t
    Jun 16, 2009 at 22:04
2

You can get an iSCSI interface for Netgear's ReadyNAS range. I think the Pro supports it natively but you can get a plugin for the lower end systems.

If you don't want special hardware, you can use a bog standard box, use Starwind as an iSCSI target and connect that way.

1
  • I must say, their XRaid is interesting, with it they seem to offer the same technology that made Drobo popular. This test on SmallNetBuilder reveals it to be best of class: smallnetbuilder.com/content/view/30725/75 Now the only problems are worldwide support and the non-rackmount formfactor, that could be a tough sell. Very interesting nonetheless.
    – w00t
    Jun 16, 2009 at 19:44
2

If you really want cheap I'd look into DRDB along with your favorit Linux distro and iscsi-target in combination with linux-ha.

All you need is time, depending on how expensive that is four you the next best thing that provides ISCSI along with RAID, but not highly available (I think they don't have DRDB or linux-ha option) is probably OpenFiler (Linux based) or FreeNAS (FreeBSD based)

Next in the row of really cheap is probably stuff like the Drobo Pro or other ISCSI enable "SOHO storage devices".

All of those options are cheaper than the classic storage providers - at least for the initial investment. Would be nice if you could elaborate what your definition of highly available is thou

2

Well, I have a Nexsan SATABeast and I can highly recommend it. Extremely stable, fully redunant (with 2 controllers) and the lowest price/TB if you need FC or would like to keep it in your existing FC SAN.

It's true that they dont have engineers all over the world, but they provide excellent and quick response by mail/phone. The closest engineer for me is about 200 metric miles away (half a business day away w/airplane), and it has so far worked out excellent for me.

Edit: The controllers also have iSCSI

2
  • well the problem we ran into in Australia was that one of our support guys had to have his credit card charged before they could ship a replacement drive, they would refund the money as soon as they received the disk back. Pretty tough for enterprise-level support :-/
    – w00t
    Jun 16, 2009 at 21:48
  • That's pretty harsh. Are there no resellers nearby who can provide disks?
    – pauska
    Jun 16, 2009 at 22:40
2

You might want to check out Scale Computing. They have a clustered storage solution that will do iSCSI, NFS and CIFS at the same time. The low-end is a 3TB cluster, and you can scale it 1 or 2TB at a time, up to something like 2PB.

1
  • Very interesting! They only have partners in the US though, so it's a no-go for us :( I wonder if VMWare will ever use this approach, every node you add to a VMWare cluster would add clustered storage.
    – w00t
    Jun 22, 2009 at 6:33
1

iSCSI san do you? We're getting good results out of the Dell Equalogic. Still maybe not what you'd call "cheap" but from what I can see, definitely cheaper than the same sort of thing on FC.

1
  • That's EqualLogic. equallogic.com I recently went to a demo about these things, they look amazing, and easy to setup.
    – reconbot
    Jun 11, 2009 at 13:04
1

Look for Direct Attached Storage using 10-15 SATA drives. Something like the Dell Powervault MD1000. It expands to 15 drives per enclosure and you can use up to 6 of them. With up to 1TB per drive it should be enough to keep you going for years to come.

1
1

A super cheap SAN is the Dell MD3000i Line you can expand them with the MD1000 storage head. I think it maxes around 28 TB with the 1000 and 7200 RPM 1TB drives in a RAID 5 config (15 drive bays in each enclosure) for sub 40K probably. It's an iSCSI solution and has served us well. Right now if you want faster drives they go up to 600GB 15K drives and 750 GB 10k Drives.

2
  • I looked into these. I disliked their crazy licensing terms. Extra costs for connecting over X number of machines, or if you had more than Y number of slices. Not cool. Jun 17, 2009 at 2:47
  • I havn't run into those. I've run into stupid things like if you hook it to vmware or more than 4 hosts they try and make you do the "Professional onsite install" but i just tell my rep i don't need it and he takes it off. Then again we have a giant enterprise account and I can get my rep to do pretty much what i want so YMMV.
    – Zypher
    Jun 17, 2009 at 3:15
1

In terms of your requirements, pretty much any supplier will be able to get you a system which will hit each of your bullet points.

Obviously you're worried about price, which means you're not looking at EMC, NetApp, etc, but there's still a lot of choice.

If you want all the functionality of the high-end arrays, look at the Falconstor NSS SA storage appliances, they're a lot cheaper than the EMC's of the world but still have pretty much every feature you could want. Otherwise look at the Dell EqualLogic PS5000E iSCSI Array if you want comprehensive support, scales up to 200TB of storage, performs well, and will do all the tasks you've listed above.

http://www.falconstor.com/en/pages/?pn=NSS http://www1.euro.dell.com/uk/en/business/Storage/dell_ps5000e/pd.aspx?refid=dell_ps5000e&s=bsd&cs=ukbsdt1

Ewan

1

Re running (Open)Solaris on an x4540 I had some experience last year in running it as a cheaper iSCSI target and it did not go according to plan: http://scaleordie.com/2008/07/06/attempts-at-using-a-sunfire-x4500-thumper-as-an-iscsi-san/

Chances are the code has matured a bit since -- that's in substance what the sun support engineers told us back then: you have to wait until the code gets better.

1
  • good writeup in your link, interesting!
    – w00t
    Sep 15, 2009 at 11:14
0

There is a company that I’m sure you will found interesting, the StarWind software company.

It’s a global provider of storage virtualization and Open iSCSI SAN solutions.

StarWind will allow converting Windows PC into an iSCSI SAN that you can use to share storage as well as provide disaster recovery. The idea is that you can take any existing Windows PC, install StarWind on it and use it's storage as a SAN.

www.starwindsoftware.com

0

I personally adore HP P2000 class SAN's. They're intro level, and give you the flexibility to use FC, iSCSI, or both depending on the controllers you want to outfit it with. You can also trick them out for storage (larger slower disks, up to 36TB/2U shelf) or performance (lots of fast, high IO disks - 24x 15k disks/shelf). They're pretty robust, and they can be scaled up on the fly by adding additional chassis. They're also pretty fully featured, though lacking some of the super fancy features you'd find on really high end storage. They may be worth checking out - we use them for a number of projects here because they're so flexible. The only catch is you'll have 1 or 2 controllers handling all of your storage, which can be a bottleneck if you want to run tons of fast storage behind it.

Not the answer you're looking for? Browse other questions tagged .