Before I ask my questions, here is some background info that may or may not be useful:

For the first time I find myself needing a DAS solution. My priority is data through-put in a single direction. I can write large blocks, and I don't need to read at the same time. The server (the data producing device) is not really a typical server, its a very powerful single board computer. As such I have limited options when it comes to the add-in cards I can install since it must use the fairly uncommon interface, XMC. Currently I believe I am limited PCIex8 gen 1 which means that the likely bottle neck for me will be this 16gbps connection.

XMC Boards I have found so far offer the following connections:

a) Dual 10GBE ethernet controller, total throughput 20gbps

b) Dual Quad SAS 2.0 Connectors (SFF-8XXX) HBA (no raid), total throughput 48 gbps

c) Dual FC 8gb HBA (no raid), total throughput 16gbps

My questions for you guys are:

1) Are SAS and/or FC, and by extension their HBAs, standard enough that I could purchase a Dell or Aberdeen storage server with a raid controller that has external SAS or FC ports and expect that I can connect it to my SAS or FC HBA, be presented with a single volume (if I so configured the storage server), all without having to check for HBA compatibility?

2) On a device like a Dell PowerVault (either DAS or NAS) is there an OS on it to concern myself with, or is it meant to be remotely managed? Is there a local interface in case I cant remotely manage it (i.e. if my single board computer uses an OS not supported by Dell OpenManage). Would this be true of nearly any device which calls itself a DAS?

3) If I purchase some sort of Supermicro storage chassis, installed a raid controller with external connections, is there a nice lightweight OS I can run just for management of the controller? Would I even need an OS since the raid card would be configured pre-boot anyway?

4) It is much easier to buy XMC based 10gigabit ethernet cards (generally dual port). In what ways would I be getting into trouble by using iSCSI as a DAS are direct cabling with SFP+ cables?

Thanks in advance


ad 1) There are different kinds of storage solutions on the market - you would need to take care what you're buying:

  • JBOD enclosures with passive backplanes - you will find some of these for SAS. They just provide cut-through connections for multi-lane links and do not have any logics or any kind of management
  • JBODs with (SAS or FC) expanders - these are more common for higher-density applications, the expander will fan out 1 or 2 (multi-lane) links to a larger number of drives. Management (mainly for zoning) might be out-of-band (through a separate management controller with IP connectivity) or in-band (SES), depending on the implementation.
  • storage appliances with own controllers - might come in any combination of FC, SAS or iSCSI connectivity (in the latter case sometimes with NAS functionality as well), typically extensible with additional JBOD shelves and available with redundant controllers. The controllers are built-in, management is out-of-band. Many solutions are using the same base chassis and a different set of controllers or add-on interfaces depending on the type of host interconnect required

The storage appliances are in general compatible to what is claimed - a SAS interconnect can be hooked up with a SAS HBA and present a single volume (of a configurable RAID level), the same goes for FC and iSCSI. The JBODs are more tricky - especially if they were designed as expansion shelves for a storage appliance, you would have to take special care to make sure they would work with (and be supported on) the chosen controller.

ad 2) of course there is an OS on storage appliances. But as it is provided together with the appliance, you can regard and handle it as "firmware". You certainly do not need to stare at the startup screen and press ctrl+alt+del before using the appliance (although you will find Windows Storage Server based devices on the market). As management typicaally happens out-of-band, you would have the option to use an additional computer for maintenance and monitoring purposes.

ad 3) If I get you right, you want to set up a RAID controller to connect your disks and to serve as a target to your SBC's HBA. I am not aware of any cards which would support this mode of operation. Expanders work somewhat in this way (you can connect targets and initiators to any given port of an expander), but you still would have to have the RAID logic on the controller of your SBC.

ad 4) you probably would not get into trouble all that much. iSCSI is pretty standard storage interconnect nowadays, performance is decent and multipath and failover options are worked out well. However, as the iSCSI stack is more complex and has more dependencies than, say, SAS you need to know what you're doing. Also you typically are in for a higher latency compared to SAS or FC interconnects:

latency summary for different interconnects

| improve this answer | |
  • Great response, thanks. In response to your answer to q3, yes that's exactly what I was looking for. It appears that Dell PowerVaults, when fitted with a PERC H800, provide this functionality (see the H800 section here dell.com/us/business/p/powervault-md1220/pd). Would the H800 be compatible with any generic SAS HBA? If I wanted a device like that what terminology should I use to search for it? How do those devices perform the task, in a proprietary way? I'm starting to think I should stick with iSCSI 10gb and avoid the whole issue. – Jay Jun 2 '14 at 2:35
  • @user222182 the MD1220 tech sheet does not say so explicitly, but it appears very much like an enclosure featuring a SAS expander. The H800 is meant to be placed into the host, so you could attach a number of MD1220 enclosures to its external ports. But you still can't use the H800 as a target, i.e. you cannot just connect another HBA to it and hope for seeing a logical SAS RAID device. iSCSI should be a decent choice if your application does not mind the additional latency. This likely is the case if it only involves few synchronous write operations. – the-wabbit Jun 2 '14 at 8:27
  • If all you can attach to your SBC is a simple HBA in IT mode (and not a full-fledged RAID controller) and cannot (or do not want to) implement the necessary volume management and fault tolerance in your OS software stack, you definitely are looking for a storage appliance (keyword: SAN) which would expose logical devices to your host over an interconnect of your choice. Try looking at NetApp's Engenio products (E2700), HP's MSA devices or Dell's MD3 product line to get a better idea of what features are available. – the-wabbit Jun 2 '14 at 8:35
  • I specifically asked Dell about where the H800 was meant to be placed, they assured me it was meant to go in the enclosure. My guess is that there was a miss-communication. The MD3 seems to offer what I want and comes in SAS, FC, and iSCSI. It seems that no matter what company it is, and even looking for appliances, its still rather difficult to tell if the device does what I want. When I am looking at an iSCSI devices its typically very obvious if it does what I want, since I've never seen a unit which didn't include raid. Is this typically the case for FC devices as well? – Jay Jun 4 '14 at 1:35
  • @user222182 if they are sold as storage appliances, it is. There are FC-connected shelves (e.g. NetApp DS14) which do not have a RAID controller but a protocol bridge (to SAS / SATA) built in, but you only will see them as add-ons to existing storage appliances. Keep in mind that FC has become pretty outfashioned lately - if you only have short distances to cross, you are looking at SAS interconnects, otherwise it is iSCSI or FCoE. – the-wabbit Jun 4 '14 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.