2

I've done a fair bit of troubleshooting but I'm completely at a loss what could be going on.

Hardware / platform

  • Supermicro X10SRi-F motherboard
  • EVGA 850W G2 PSU (one of the highest rated for power quality at Jonnyguru.com)
  • 128GB Crucial DDR4 RDIMM
  • LSI 9211-8i PCIe HBA flashed to latest P20-IT (from LSI/Avago website)
  • 8087-to-quad-SAS connector cable (new: see this picture, the kind of cable where the power side accepts a SATA PSU connector)
  • Seagate 6TB SAS drive (new: ST6000NM0054)
  • Various other Seagate 3TB - 6TB SATA drives (test purposes)

Problem / troubleshooting so far

This is a new server being set up, so all components are new, although some have been tested already before now.

On booting, the HBA didn't recognise or report the 6TB SAS drive (either via the main BIOS or via its own OROM -> SAS topology) and the 6TB drive was cold and not spinning up. No other drives were connected. The rest of the system works fine, so on the face of it the issue is limited to one or more of bad HBA, bad cable, or bad drive.

Troubleshooting steps so far:

  1. Connected 6TB SAS drive using different terminator on the quad cable, and the quad cable to both 8087 ports. No change - implies the issue isn't one specific terminator or port.
  2. Connected various Seagate 3TB-6TB SATA drives using same cable (same manufacturer and similar modern range to eliminate subtle compatibility issues if any). All recognised, reported, and spun up perfectly as normal on boot, on both 8087 ports and on all 4 terminators, and over multiple reboots - implies HBA and cable both work fine, at least for SATA. (Would be odd if they worked perfectly for SATA but not SAS.)
  3. Kept identical connections but replaced SATA drives by 6TB SAS drive, not changing anything else. As before, 6TB SAS drive wasn't recognised or reported by HBA, and wasn't spun up.
  4. Tried exactly the same with a different card and platform m- LSI 9260-8i RAID controller on an ASUS based desktop. Again all SATA drives immediately recognised and spun up, but 6TB SAS drive isn't/doesn't.
  5. Reluctantly concluded that however unlikely, the most likely issue was 6TB SAS drive DOA and RMA'ed it. ("Reluctantly" is because I've never actually had a DOA before, the drives are usually reliable, and if it is dead then by far more usual/expected would be that it's at least recognised but non-functional. I just couldn't figure a more likely issue than complete DOA.)
  6. Just received the warranty replacement - and getting exactly identical symptoms with the replacement as well: (a) When the 6TB SAS drive and any SATA drive are connected to 2 terminators and the system boots, the SATA drive is immediately recognised, reported and spins up, while the 6TB SAS drive stays cold and still. (b) When the 6TB SAS and any SATA drive are connected to the 9260-8i RAID card in the other ASUS desktop the SATA drive is likewise immediately recognised but the 6TB SAS drive stays cold.
  7. As a last step, re-re-read the 9211-8i HBA user guide in case I missed anything first time, and re-checked the BIOS. Can't find anything that would seem to explain this, or any statement that SAS drives will not be recognised unless/until <some action/content>.

Didn't really believe it was DOA first time. Definitely don't believe it's a DOA now. But if not, then what is it, and what can I be missing?

I've tested everything (AFAIK) in the component chain, the HBA just doesn't have much OROM interface that can go wrong, or any options to recognise SATA/SAS/both, or anything like that, and the main PC/server in both cases just leave detection to the HBA/RAID card. I've tested on two completely different platforms, with two different models of controller cards, with SAS vs SATA drives, and I'm utterly stumped.

(Note: I'm slightly limited as I'm starting to transition from SATA to SAS, with the intent being to replace SATA by SAS as they wear out, so at the moment I don't have any other SAS disks or cables to test with, which I would otherwise have done too. But I think I've probably covered that by testing the cards+cables while varying SATA/SAS)

Updated for more accurate title to help others, now more info obtained. See answer.

  • What is exact type of the SAS drives do you have? 9211-8i is based on LSI 2008, which supports only drives with 512n blocks. If you have drives with 512e or 4kn you need a HBA with LSI 2208/2308/3008/3108/... – Yarik Dot Nov 22 '16 at 22:56
  • Added to question. It's an ST6000NM0054, apparently it's 512e. But so are my other SATA drives... the same 9260-8i that won't spin up the ST6000NM0054 (6TB, 512e) has never had an issue recognising and spinning up, among others, my old ST4000DM000 (4TB, 512e AFAIK). It sounds plausible so I'll test this out with other 512e drives and check the sectoring more closely and comment more tomorrow if I find anything different. – Stilez Nov 22 '16 at 23:18
  • docs.avagotech.com/docs/IT-SAS-Gen2.5CompatibilityList lists a variety of SAS 12g 512/512e/4k drives as compatible with the HBA, as best I can make out? – Stilez Nov 23 '16 at 1:39
3

I spoke to LSI (now Avago) tech support for storage, in Germany. They considered that if 2 different kinds of "known good" controller in 2 different machines both recognised all sata but not this sas drive (on any port and connector) then it was very likely the drive.

They also suggested a further test - to connect the power side of the drive only (NOT HBA/motherboard/data wires) and turn on the server. (He warned me it would "sound crazy"!) Apparently like SATA, SAS drives spin up when they are first powered if the data side isn't connected (I didn't know that, wonder how staggered start works then?), providing a very good test that relies only on the PSU and power feed to the drive, and nothing else.

Sure enough, SATA drives all spin up, this drive doesn't. He felt that was enough to be "almost certain" its a 2nd bad drive, however unlikely, without spending cash. The serial number was also almost identical to the original dud drive (1 digit change); so he also suggested speaking to the manufacturer and raise the question if they have any other similar reports for this drive, as it could be a bad batch.

Update April 2017:

I thought for a while that the issue was the LSI 9211 BIOS needed to be disabled, based on an online thread. I disabled the bios and it did work... but later when I moved the box it stopped working and I couldn't figure why. I took this info back to LSI tech support and they said it wasn't possible that the BIOS could be the issue or disabling it could help. They felt that moving the box was likely to have undone a fortuitous cable working and turned it back to not working.

They said to try a new "forward" or "fanout" cable, and specifically, Adaptec (on the side, as it's a competitor!!) which are more reliable than most for SAS. They said that it wasn't always clear or marked whether a cable was the right kind or not, and to check carefully.

The exact SAS cable one would need will vary depending on what interface the HDD and card have. The 9211 has a SFF-8087 connection and my HDD has an SFF-8082 connection (looks a bit like SATA but power and data ports joined).

I was dubious about it being the cable (since the cable did work fine on electrically similar SATA), but went ahead and contacted Adaptec who commented that getting the cable right can be quite challenging in the sense of being sure exactly which kind of cable is needed. They checked the card specs and HDD specs and recommended their 2275300-R on Amazon, and much to my amazement it worked first time, so I guess they must have known what they were talking about.

  • 1
    You should use a different cable just to be sure, as I've seen similar issues with chinese 8087s.. – Peter Zhabin Dec 23 '16 at 14:35
  • Same cable worked fine with the same HBA and a data drive though? – Stilez Dec 23 '16 at 23:17
  • 1
    I should reverse my comment above - despite the unlikeliness, the cable apparently was the issue. See update in Q. – Stilez Apr 30 '17 at 10:48
0

Actually the real cause for this grief is due to changes in the SATA / SAS power standard. Pin 3 is the issue.

Using a molex style power will fix this problem when using newer SAS drives (built after 2016).

So it didn't matter if it's cables (unless it uses molex for power), HBA or drives it all points to this. Fun huh?

https://www.hgst.com/sites/default/files/resources/HGST-Power-Disable-Pin-TB.pdf

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.