Cheapest way to connect 20-24 Sata II HDDs in a budget storage server? [closed]

Question

I need to assemble a high density storage server for as cheap as possible.

It's been a while for me and the last systems I integrated didn't even have Sata yet...

During my Research I of course stumbled about Nexsan SATA Beast, the BackBlaze storage Pods as well as some ridiculously overpriced HP Proliant or Dell storage solutions.

Finally I choose Norco cases as the way to go.

My eye is set on the RPC-4020, which is a 4U 19" Rackmount case with 20 Hot Swap 3.5" SATA/SAS Hdd trays (Backplanes included) and room for two 2.5" OS drives as well as a Slim Line CD-Rom. The backplanes connect with a single SATA port for each drive, so there are 20 internal SATA ports to to be connected. They also have redundant power ports which I think is quite nice. The cheapest price I have found is 290$ + 40$ shipping. In europe the cheapest unfortunately is 370€ (500$) + 40 € shipping...

A nice alternative would be the RPC-4224 which has SFF-8087 Mini SAS connectors that bundle 4 SATA trays each. But it doesn't seem to be available in Europe (where i am) anywhere.

So here comes my problem:

What Mainboard/Controller to choose to connect them for as cheap as possible while still having nice data rates? I have to say that the server is intended as a Storage server with 1gps connectivity and the data transfer will be distributed very evenly across all drives. I also don't require any raid functionality. This is all done at application level, I just need JBOD.

So for example if I go for the RPC 4020 Model I need to connect 20 Storage + 1 OS + 1 CDROM Sata ports.

I searched a bit and stumbled across this very low priced controller:

http://www.intel.com/products/server/raid-controllers/SASWT4I/SASWT4I-overview.htm

They sell it for 115 € here and the specs say it can control up to 122 hard discs and has 4 Mini SAS connectors.

So I would use 4 Mini SAS 36pin - 4 SATA 7pin cables to connect 4 SATA drives to each port and choose a Mainboard taht has 6 SATA on board (for example this one) and hurray, I can connect my 22 SATA devices for as low as about ~ 220 EUR (cpu, ram, psu, case not counted)

Question: WOULD THAT WORK? And if not, why?

2nd Question: If I go for the 4220 or 4224 Model, I have internal Mini SAS connectors. Am I right in assuming that the backplane than acts as a "SAS Expander"? And can I just plug these SAS connectors into any SAS port I can find on my controller / mainboard or are there certain requirements? I know that SATA port multipliers only work with controllers that are ready for that. But isn't this expansion already implemented in the SAS standard?

I am sorry that this is a very broad question, but I really spent the last week reading up and it seems to be not so clear! Especially all the controlling hardware specifications!

3rd Question: A lot of hardware specs feature "internal channels" and "internal connectors". The connecors are the physical numbers of places where I can plug a cable in. I got that. But are the "internal channels" always the maximum numbers of physical drives that can be used in the end? Or can I enhance this further by Expanders/Fanouts?

4th and last question: What do you think about the setup so far? Do you know any good alternatives? Maby I am completely going the wrong way and some DAS would be way better? Are there any comparable chassis available in europe? Please feel free to say whatever you think is relevant to the subject!

EDIT:

For all of you who are intersted in what I finally came up with:

1 x Codegen 4U-500 Case / ~65€

1 x MSI 760GM-P33 Board (6 onboard SataII) / ~40€

3 x DeLOCK 70137 2xSataII / PCIe controller / ~15€ per piece

1 x Evertech ET-3612 2xSATA / PCI controller / ~15€

1 x Athlon II 2x3GHZ / 65w CPU / +50€

1 x Standard ATX PSU / +20€

3 x HDD 3.5 -> 5.25" spacers / ~2€ per piece

1 x 2 GB Ram / ~15€

I read some bad review about the Delock controllers and the delcok website also said they needed patching to support Jbod, so I decided to buy only one for testing. But they all worked like a charm. Also booting from them was no problem at all.

The case itself has room for 10x 3.5" drives, additional 3x5.25 make room for a 4x3.5" hdd cage witha ctive cooling. Makes a total of 14 drives which is perfectly served by the 6+2+2+2+2 ports. System setup fast and easy per USB, no need for an optical drive theese days.

What I did however to save some money was not to use an HDD cage inside the 5.25" bays but to use standard spacers to fit in 3x3.5" drives. They have a lot of space between them and with two nice rear fans dont need extra cooling. The one drive that wont fit is mounted on the carrier with the slot lockers that you can see here (i is "dangling" from top, only fixed on one wide with two screws but thats fine):

The mainboard is only Micro ATX so there would be enough room for two drives on the side but i dont have sata ports for that. I could have used one 4 port controller but the value is not so good then.

My OS drive is an internally mounted USB stick.

I needed some power splitter and sata cables which i had lying around but all in all we are looking at a complete system for about 260 € that makes room for 14 drives.

Answer 1

It sounded like your basic description would work. I currently have a Norco 10bay 3U with built in 2x SFF-8087. I've been looking to expand to case similar to the 24bay one you found, as I've already got 2x arrays of 5 drives.

The PC is a straightforward Core i3 setup with a Supermicro AOC-USAS2-L8E as the controller. I use Solaris (or you could use FreeBSD depending on your drivers) and have a Zpool of 2x RaidZ arrays. On my gigabit network we get read speeds ~115MB/s and writes around 90MB/s

A note about ZFS, it can be a memory hog (think at least 4GB), but it's super fast, reliable, easy to expand, and has some nice extras.

Answer 2

The Supermicro 846E16-R1200B http://www.supermicro.com/products/chassis/4U/846/SC846E16-R1200.cfm may be a better choice as it has an lsi sas expander on the backplane and will allow you to deploy a 4 port SAS RAID card like an LSI 9260-4i or LSI/3Ware 9750-4i instead of a high port count card that costs a lot more. You can still go with SATA disks if that is your preference. Enterprise SATA is fully supported. Desktop drives, however, are limited in support as the RAID card manufacturers have chosen to take them off of their menu.

Answer 3

Have you considered to use cheap 1U or 2U servers ?

As this kind of server is very common and as you will buy lots of them, you may be able to get them at a very good price.

There will be less data on each server, the loss of one will be less important and putting a new machine into the cluster will be quicker.

It seems unlikely that one 1Gb/s network card will be enought to grant access to 20 or 30To of storage, if you split the data accross several servers, the global bandwith of your storage will be better.

It is easier to scale up with small machines (put one more at a time, adjust CPU and RAM) than with one big, also a single failing part is harder to diagnose and repair on a big machine with lots of different parts...

The price of one big machine is not necessarily higher than the price of serveral small because big configurations are uncommon, need special enclosures, lot of extention cards, big motherboard with a lot of RAM slots, multiple power supply ...

Answer 4

Based on your application-layer redundancy, I'd make a strong recommendation for GlusterFS. The whole point of Gluster is to create redundancy through lots of cheap storage, even without RAID. Putting together many Norco boxes with individual drives configured properly (with redundancy across servers) is a reasonable way to go.

Gluster scales very nicely for speed and capacity, and the latest versions even support changing your redundancy and striping in a live deployment. Gluster has a page showing those who care to mention their deployments, some of which reach into the petabytes.

Caveat: I've only created small deployments of Gluster, so depending on your scale you may want to ask someone more informed.

Answer 5

To answer question 4 only: I suggest that for large storage volumes, that you should use Network Attached Storage (NAS). There are off-the-shelf NAS appliances that cost only a few bucks more than the hard disk that they contain, and they offer very good performance when compared to internal SAS or SATA drives.

RAID example: This is one of many ways to use cheap off-the-shelf consumer level NAS appliances as a RAID. In theory, you could use them for anything that you could use a local block device for. (You could even skip the part about making the RAID and use the loop devices directly.) I can't see that there is a limit to how many devices can be used like this. I expect that this should scale to the thousands, some number geek should test this...

192.168.0.10-12- the IP numbers of the NAS appliances.

mkdir /nas /nas/unit0 /nas/unit1 /nas/unit2 /mnt /mnt/raid

mount 192.168.0.10:/files /nas/unit0

mount 192.168.0.11:/files /nas/unit1

mount 192.168.0.12:/files /nas/unit2

cp /dev/zero /nas/unit0/volume0

cp /dev/zero /nas/unit1/volume1

cp /dev/zero /nas/unit2/volume2

losetup /dev/loop0 /nas/unit0/volume0

losetup /dev/loop1 /nas/unit1/volume1

losetup /dev/loop2 /nas/unit2/volume2

mdadm --create -l5 -n3 /dev/md0 /dev/loop0 /dev/loop1 /dev/loop2

cat /proc/mdstat

md0 : active raid5 loop2[2] loop1[1] loop0[0]

  3416798208 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

mkfs.xfs /dev/md0

mount /dev/md0 /mnt/raid

df /mnt/raid

Filesystem Size Used Avail Use% Mounted on

/dev/md0 3.2T 0.1T 3.1T 1% /mnt/raid

Answer 6

You can use this for enclosure http://www.pc-pitstop.com/sata_enclosures/scsase246g.asp. I think you cannot find anything cheaper and still of decent quality. I recently built budget array on their 12-bay version myself. I attached the enclosure to a 2U rack mount NAS server built from components for around $2000 including SLES11 license. Here is the server configuration:

• Case – iStarUSA D-213-MATX (~$70)
• PSU – Antec EA-380D (~$60)
• MB – Supermicro X8SIL (~$165)
• CPU – Xenon X3430 2.4GHz (~$220)
• Memory - 4GB (~$85)
• Boot disk - a small SSD (~60-100)
• RAID – Adaptec 5085 + ABM-800 BBU (~$1000). I also tried LSI 9280-8e but run into problems.
• OS - SLES11 (~$350)
• SFF8088 <-> SFF8088 cable (~$50)

I've chosen hardware RAID for easy of configuration, but you can go with software raid (although I do not recommend) and it will save you another grand.

Answer 7

If you want to do DAS, I would take a look at a setup like this: Cheap server case (are you sure you need hotswap?), HP expander board, adpater for power to the expander boardm and a PSU.

Something like this: http://www.servethehome.com/sas-expanders-build-jbod-das-enclosure-save-iteration-2/

You aren't going to get a cheap NAS pushing 1GB unless you aren't paying for the NAS OS, such as Openfiler.

Answer 8

Note: I'm the mostly-happy owner of a Norco-4224.

Question 1: Yes, this will work. You will need what's called a "reverse breakout cable", though, to connect your regular on-motherboard sata ports to the SAS backplane used by the Norco: http://www.amazon.com/Discrete-SFF-8087-Reverse-breakout-cable/dp/B002MK7F0Y

You cannot use a standard SAS --> Sata cable for what you're trying to do. It must be the reverse breakout cable. All drives my 4224 are currently populated in this manner.

Question 2: They are not really SAS "expanders." Think of a single SAS channel as 4 Sata channels (electrically speaking). SAS has the capability to also be "switched"/expanded, somewhat like how you sometimes see eSata use port-multipliers (1 port --> 5 ports), with some performance trade-off.

The ports on my 4224 are just straight SAS ports. You could potentially get a SAS expander module to work w/ them (if you only had a single SAS cable, etc), but I've crunched the #'s: for home use, it's too damn expensive. The sata port --> reverse breakout --> sas connector, or even just paying for a multi-sas-port card is more cost-effective. The expanders are ridiculously expensive, and really only used when you're looking to fill things like a 42U chassis of disks.

Question 3: not sure I fully understand, but I think you're getting confused by the different notions of SAS channels and expansion, and how they interrelate.

For home usage, easiest thing to think of is: don't expand, and 1 SAS == 4 Sata. =)

Anyway, sorry you wound up going away from Norco, I've been pretty happy w/ mine (aside from having to replace insanely loud fans.)

Cheers!

Answer 9

The Intel SASWT4I is a rebranded LSI SAS3041E adapter.

http://www.lsi.com/storage_home/prod.../lsisas3041er/

Features - 3Gb/s per port - (4) internal SATA x1 connectors - Integrated RAID 0, 1, and 1E - 4-lane 2.5Gb/s PCI Express - MD-2 small form-factor design - Connects to both SAS and SATA HDDs and tape drives - Allows more than 122 total end-point devices - Fusion-MPT architecture providing more than 140,000 I/Os per second - Supports all major operating systems

This controller has 4 SATA ports and hence will only directly support 4 drives via SATA cables. If you intend on using more than 4 drives, you would have to use a reverse SATA to SFF-8087 breakout cable to connect to a SAS expander which in turn can connect to more drives.

I think you would be better off looking at a different controller. There are other alternatives in the same price range that give you at least 8 SAS/SATA ports via 2 SFF-8087 connectors.

(taken from http://hardforum.com/)

Answer 10

thanks for the tip. but still quite expensive i think. thing is i dont requrie high quality or high availability. this is all done at application level

Is that to say that if your data goes kaput, everything is ok? Going ultra cheap on data storage is the reason why a lot of IT folks get sacked. Having a short term outage is very minor compared to data loss.