Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm new to SAS expanders and I'm trying to get things right. I understand that SAS expanders, when used with SAS controllers which supports expanders, allows you to utilise the full capacity of supported drives on a controller.

However, say I've got a SAS controller with 4 x SSF-8087 ports, which by itself would support 16 disks. I could then, as far as I understand, instead connect each of these ports to an expander to increase disk capacity.

However, will the SAS controller "see" all disks as if they were connected directly to the controller and not through an expander. Could I look at controllers as switches in a network? Would I be able to create a RAID from a disk on expander 1 and expander 3?

UPDATE:

I don't really have a specific project where I need to use SAS expanders. I'm just curious about how they work, and was thinking about how it would work if I used one or more SAS expanders with a LSI MegaRAID controller (9260-16i) which is running in one of our servers. However, I haven't really put any thought into the distinction between "local" disks and disks in another enclosure. The LSI MegaRAID controller has 4 SFF-8087 ports, which makes it capable of accomodating 16 drives.

share|improve this question

1 Answer 1

up vote 7 down vote accepted

First, stop and think about your realistic goals...

The short answer to your question is that SAS is a switching protocol. The drives connected via expanders are all accessible and can be addressed transparently by a single controller.

However, the rest of your questions are going to be a function of your particular controller, your storage architecture and the backplanes in the storage enclosure(s) you end up using.

It's not possible to give a specific answer without more information.

  • How many disks do you need?
  • What RAID controller or SAS HBA are you using?
  • Will these disks be internal or external?
  • What specific SAS expander card are you looking to use (it's not like there are many on the market)?
  • What server and enclosure hardware are you considering?
  • What is the application/reason for this setup? (ZFS? Storage Spaces?)

Let's use your example of a controller with 4 x SAS SFF-8087 ports. These are meant for communication inside of a chassis. Each SFF-8087 port is a 4-lane connector that can accommodate 4 drives. 16 drives total, in your case... Servers typically only have 4, 8, 12, 16, or 24/25 internal drive bay capacities.

  • At the 4 and 8-drive level, the internal server drive backplane will have a 1:1 lane-to-drive ratio and typically have one or two SFF-8087 ports. No oversubscription!

  • For 12-port servers, the expander is usually built into the backplane, and the input will be a single SFF-8087 port. Oversubscription!

  • For servers with 16 internal bays, you may see four separate SFF-8087 ports intended to be served by a single or multiple SAS/RAID controllers. But at 16-ports, you may have an expander on the backplane instead.

  • Almost every 24 or 25-bay server I've encountered has expanders built onto the drive backplane and accommodate one or two SFF-8087 ports. The exception is something like the HP ProLiant DL370 G6, which could accept three 8-bay drive cages, each requiring 2 SFF-8087 ports.

So when do people use PCIe card SAS expanders?

Note: HP makes a great PCIe SAS expander. There's an entire community built around its use for storage projects.

The most common case I see is a motherboard or PCIe controller with TWO SFF-8087 ports and a need to connect more than 8 drives split across multiple drive cages. I can't envision something like your scenario where you have 16 ports available on a controller and would want to multiply that via expanders to accommodate more internal drives. If you're working with a server with more than 16 disks, chances are that you'll be encountering a backplane expander somewhere. This would make the PCIe expander unnecessary.

This is all different when you look at external storage, though. The transport there would be a SAS SFF-8088 external cable. Most external drive shelves/JBODs have expanders built-in.


Example RAID configuration output from an HP Smart Array P410i controller connected to two drive cages inside of an HP ProLiant DL370 G6 server through an HP SAS expander card. Note the card is represented as an enclosure (Vendor ID HP, Model HP SAS EXP Card) and has its own SAS WWN.

Smart Array P410i in Slot 0 (Embedded)    (sn: 5001438006503640)

   array A (Solid State SATA, Unused Space: 0  MB)

      logicaldrive 1 (223.5 GB, RAID 0, OK)

      physicaldrive 5C:2:8 (port 5C:box 2:bay 8, Solid State SATA, 240.0 GB, OK)

   array B (SAS, Unused Space: 0  MB)

      logicaldrive 2 (68.3 GB, RAID 1, OK)

      physicaldrive 7C:2:1 (port 7C:box 2:bay 1, SAS, 72 GB, OK)
      physicaldrive 7C:2:2 (port 7C:box 2:bay 2, SAS, 72 GB, OK)

   array C (SATA, Unused Space: 2651025  MB)

      logicaldrive 3 (1000.0 GB, RAID 1+0, OK)
      logicaldrive 4 (500.0 GB, RAID 1+0, OK)

      physicaldrive 2C:1:1 (port 2C:box 1:bay 1, SATA, 1 TB, OK)
      physicaldrive 2C:1:2 (port 2C:box 1:bay 2, SATA, 1 TB, OK)
      physicaldrive 2C:1:3 (port 2C:box 1:bay 3, SATA, 1 TB, OK)
      physicaldrive 2C:1:4 (port 2C:box 1:bay 4, SATA, 1 TB, OK)
      physicaldrive 3C:1:5 (port 3C:box 1:bay 5, SATA, 1 TB, OK)
      physicaldrive 3C:1:6 (port 3C:box 1:bay 6, SATA, 1 TB, OK)

   Expander 250 (WWID: 50014380133153E6, Box: 1)
   Enclosure SEP (Vendor ID HP, Model HP SAS EXP Card) 248 (WWID: 50014380133153E5, Box: 2)
   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 249 (WWID: 500143800650364F)

enter image description here

share|improve this answer
    
Good answer. I was going to write something similar, but you beat me to it. Additional word of warning: Putting an expander behind another expander is usually a bad idea. Both from a performance and a stability perspective: There is a huge chance for subtle compatibility issues. –  Tonny May 10 '13 at 9:11
    
@Tonny The point here is that an expander may be redundant to wholly unnecessary in the OP's situation. But without specifics... we don't know. –  ewwhite May 10 '13 at 9:16
    
You are right. His intend is not clear from the question. And most server already have internal expanders. That's why I warned putting one after the other. From experience I know that most people don't realize that these things are not really plug&play. You need equipment that is matched to each other or things can go very wrong. –  Tonny May 10 '13 at 9:25
    
@ewwhite thanks for your reply. I've updated my question to answer some of your questions. –  sbrattla May 10 '13 at 10:17
    
@sbrattla For the controller you've listed, you have plenty of internal ports. What type of server is it? –  ewwhite May 10 '13 at 10:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.