Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm working on a project with a client who recently had a large system replacement for a dedicated ERP application. I'm inheriting this hardware setup and want to understand what it means.

The server is a ProLiant DL580 G7 (4 x 8-core CPUs, 64GB RAM). The customer has a set of ~500GB of data that historically grows 1-2% per year. The system has plenty of onboard storage in the form of 8 x 300GB 6G SAS internal disks. The server is connected to an MSA2324sa SAS-attached SAN unit, fully populated with 24 x 146GB disks. This SAN has two SAS controllers and the server has two HBAs, each connected via SFF8088 cable to one port on each controller. This is the only server connected to the SAN.

While this is fine and appears to be a standard cabling setup, the confusion comes in the presentation to the OS. The server is running RHEL 5.7. Once I gained access to the system, I noticed the following df and fdisk output:

Here's a pastebin of the lvdisplay, vgdisplay and pvdisplay output.

System overview LVM setup

This seems odd to me. Unfortunately, I can't find any information to confirm the validity of this setup. Most of my experience has been with Smart Array controllers and "dumb" JBOD expansion units, but I tend to use HP's facilities to carve LUNs and manage volumes.

Looking at this setup, why was it necessary to use LVM striping? Is this uncommon? Does it have more to do with the multiple SAS connections to the server? The vdisk/volume arrangement of the MSA seems like it would provide the necessary flexibility. As-is, it seems like this SAN has no room for additional servers or provisions for spare disks, etc.

I just want to check with engineers who are more familiar with these deployments. Is there anything wrong with this setup?

share|improve this question
I was not aware of that 16-drive limit. Does that mean that the only way to leverage that amount of space is to split things up and use software RAID? Does that 16-drive limit depend on RAID level? – ewwhite Jul 26 '11 at 22:05
Thank you. Do you think there's a performance implication with the above setup? Would it have made more sense to just use a JBOD enclosure like an HP D2700, since this SAN is basically being used a direct-attached storage? – ewwhite Jul 27 '11 at 1:17
Thank you. This should have been the answer. – ewwhite Jul 27 '11 at 20:03
Moved; and now it looks like you talk to yourself. =] – Chris S Jul 27 '11 at 20:35
URGHH... SAN→ Storage Area Network. It does not mean a Storage Subsystem. (pet peeve of mine) – MikeyB Jul 27 '11 at 20:43
up vote 2 down vote accepted

The MSA2000 can only put 16 disks in most array types, hence two arrays software striped. The 16 drive limit is on all arrays except RAID50 and 60 (which is 32 drives); this is for G2 units (ref:… [end of page 15]) and G3 units might be different (I haven't had the pleasure yet). You could still use a hardware RAID HBA on the MSA's logical drives to present a single drive to the OS too.

I wouldn't think there would be much, if any performance penalty. However I would be concerned that if one cable fails the array goes down (I'm not sure of this, but suspect so). The D2700 probably would have been quite a bit cheaper to begin with, but I doubt you'd see enough performance change to justify it now.

Take all this with the caveat that I have not used a MSA2000sa, only the "i" (iSCSI, have this inhouse) and "fc" (Fibre Channel, customer site). The "sa" units are somewhat rare as the connection medium is quite limiting.

share|improve this answer
I spoke with an HP technician who said that a firmware update made it possible to allocate more than 16 physical drives in a single vDisk. He was unable to point me to the specific update that included this functionality, but he insisted that the feature is available. – ewwhite Jul 28 '11 at 21:18
Ever find this update ? We just acquired another MSA2000sa and just hit the 16drive limit using the latest published firmware. Short of a new firmware we will be using LVM to stripe 2 volumes. – Steven Williamson Jun 11 '12 at 16:22
Oh Also one cable failure should not effect access to either volume. There are two controllers in the MSA A and B, if either controller is lost (or the path to the controller) the other one will take over the vdisks it was responsible for. (Assuming your running multipath on Linux) try multipath -l from some informative output. – Steven Williamson Jun 11 '12 at 16:26
I've never seen an update that changes the aforementioned limits (16 drives in R10 or R5/6; and 32 drives for R50/60). The G2 hasn't got much attention since the G3 series came out. Using more than 16 drives in a R10/5/6 setup is highly inadvisable as the risk of dataloss during rebuild is quite high. – Chris S Jun 11 '12 at 17:55
@ChrisS The client never took my recommendation. Today, I'm seeing mptscsih: ioc0: attempting task abort! (sc=ffff8101a159a0c0) sd 1:0:0:2: messages on one of the SAS paths to the array. @StevenWilliamson, I checked multipath -l and there's no output. I don't think there's any multipath. – ewwhite Jul 1 '12 at 9:15

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.