We have a FreeBSD storage server, listing the specs:
Server1:
- 3Ware 9500S-12 RAID Card
- 12 x WD20EARS (2TB WD Caviar Green) SATA Disks connected to it via JBOD export (Not as single units due to the TLER problem with these disks
- 2 x raidz2 zpools, 1 using 5 disks, the other using 7.
- 3Ghz i7
- 16 GB RAM
- FreeBSD RELEASE-8.2
Currently we are having performance and reliability problems with the RAID card, and since we have been exporting the disks as JBODs we thought it would be better to just buy a bunch of SATA controllers to plug all the disks into SATA connectors directly rather than aggregate via the 3ware card. (modern 3ware cards like the 9650SE-12ML are about $800.. when 3 sata controllers would be about $140)
We have a number of sata ports on the motherboard already, so yesterday I tried destroying the 5 disk zpool, disconnecting the disks from the 3ware card and plugging them into the motherboard sata connectors.
When I booted up the box, I could see that the disks were seen however the console was dumping huge amounts of lines like this:
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=512
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=64
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=2
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=16
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=128
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=1
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
ad10: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
(For all the disks attached).
Checking their smart status everything was fine, when running a diskinfo on them it would recognise the disk correctly but then get read/write errors when trying to perform tests.
We tried everything we could think of yesterday, but couldn't resolve it. In the end we repluged the disks back into the 3ware card and they came up fine again.
Does anyone know why this might be happening? I would assume since we are just using straight passthrough (jbod) via the 3ware card the disk should just show up in tact? Is there anything we need to do to the disks to get them to show up?
Any thoughts/comments/help to point us in the right direction would be much appreciated.
