1

I need to build a server which is capable of capturing 20 Gigabit/second network traffic (2 x 10g network adapters - 99% utilization) and store them on a disk with zero packet-lose. The requirements is supply the ability to record 20-30 minutes of continues traffic.

Not being an “Hardware Expert”, I have search the net for today’s fastest hard drives and came up with the following option: Setting Raid-0 (stripe) over 6 x 600 GB WD’s VelociRaptor SATA drives, which claim to be the fastest (sata) drive on earth (6 Gb/s interface) - that would provide 36Gbps writing speed.

My questions are basically, having the hard drive spot sorted out,

  1. What about the other parts of the machine, what kind of a motherboard would I need?
  2. What kind of a CPU?
  3. How many CPUs?
  4. How much RAM is needed given the fact data is not manipulated but written into the hard drives?

I would like to thank you all in advance for your help and advises.

6
  • 4
    Just because the sta link is 6 Gb/s doesn't mean the drive will actually write at that speed. That is basically the speed between the computer and the cache, not the actually drive. If you check the actual spec sheet you will see that the maximum sustained write transfer speed is 145 MB/s. So those 6 drives really only give you about 7Gb/s
    – Zoredache
    Aug 10, 2010 at 17:09
  • @zoredache you are 100% right. I am going back to the drawing board (rephrasing the question) Aug 10, 2010 at 17:18
  • How long do you want to store your data? How about reading from it? Your 6x 600G drives in RAID-0 have 3.6TB of space, and you're writing out 2.5GB/s, meaning you'll only store 24 minutes (!) of data. Then, assume it all works, you'll want to read it. Well, your disk bandwidth is saturated with writes, no time to read (without losing data)...
    – Bill Weiss
    Aug 10, 2010 at 18:30
  • Mechanical drive write performance drops considerably as you reach the end of the disk. See the graph on slide 6 at cacetech.com/sharkfest.10/… Aug 10, 2010 at 18:50
  • @Bill when I write, I only writes. Read is to be done later on, but at that time, no writes is active Aug 11, 2010 at 2:59

1 Answer 1

2

While I have my doubts about how realistic this sort of thing is with OTS hardware; here's my thoughts:

Each of those drives can do about 1Gbps (max); you'd need an array of at least 20 to get the write performance you need. Realistically you'd probably need upwards of 30 drives to get continuous throughput and some kind of redundancy; though I don't think there's many SAS RAIDs that can do 2.5GBps with RAID5 or 6; so you might be stuck with RAID10.

You'll need 3 to 4.5 TB of usable storage for that much data. With the Raptor drives you'll easily have that much, even with RAID10 (which is probably your best choice here).

The CPU, PCIe lanes, and RAM shouldn't be too much consideration; other than whatever the RAID Card and NICs need. I'd get a fast CPU before one with a lot of cores. It's not doing much itself, just coordinating DMA transfers for the most part. I'd grab at least 4GB of RAM; though more is pretty much going to be better all around.

6
  • chris, I just sent you an email at comcast, see inline ;-) Aug 10, 2010 at 17:40
  • 1
    You might be able to get away with less hardware if you take a look at solid state drives. You can get get much faster the write speed with SSD, of course at a much greater cost.
    – Zoredache
    Aug 10, 2010 at 17:47
  • @zoredache, I did check that option, especially this one [1]. However, the fact I cannot build a raid from several of them, that is from all disks available at two or three pci-express boards eliminate that option [1] ocztechnology.com/products/solid-state-drives/pci-express/… Aug 10, 2010 at 17:53
  • @chris, would not raid 0 perform better than raid 10? Aug 10, 2010 at 17:54
  • @Tzury, RAID0 would be faster than RAID10; I assumed you wanted some sort of continuity in the case of failure (which is getting somewhat likely with the number of drives).
    – Chris S
    Aug 10, 2010 at 19:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .