0

In the last few weeks we started serving more and more videos by http, ranging from 50 to 500mb each, and as we didn't had a new server to use it just for that i set it up on our live streaming server.

The box is pretty decent imho (2 x quad at 2ghz, 4gb ram, 4 x 300gb sas 15k rpm in hw RAID1, running Linux 2.6, ext3 fs with deadline scheduler, nginx as webserver) but we still had issues with delivering the files, mainly i suspect because our iowait was ranging from 10 to 45% and util was around 90% most of the times, while cpu was around 20-30%.

From what i've read that should've hold our ~300-400mbps traffic, but it doesn't..

Being on a budget we can't rely on a CDN but we can get more processing power if needed and the datacenter can handle our traffic.

Now we got a new server that should do only this but i don't know what would be the best way to setup it, main questions for now being:

  • what raid level would suit this task more? now have 4 x 300gb + 2 x 146gb, both SAS at 10k rpm, hw raid, would like more space but redundancy is also important
  • how would i expand it later, lvm a good idea?
  • how much ram would it need, for now it has 16gb but can add more if it would give us a decent boost, any tips with it?
  • decided on xfs for filesystem, anything against/better suited for this?
  • how much could i rely on only this 1 server?

Any other advices are appreciated.

Thank you.

Improve this question

2 Answers 2

Reset to default
0

I am not a specialist with this sort of optimization, but here are a couple of thoughts that come to mind:

  • RAID: Serving up video, I would think you will primarily be performing sustained reads. For that purpose, I tend to think that RAID1 should be fine. A RAID 1+0 configuration might be better because there is more opportunity for parallelization - though I haven't researched the numbers on this. You may also be able to play around with the RAID settings; perhaps a larger stripe size would be better for your situation?
  • RAM: Why not start with what you have, and then monitor the usage? If you find that the server is doing too much swapping, then you can always add more RAM.

You seem to have a lot of questions, many of which might answer themselves once you throw the switch on the new server. I'd make sure that you can fall back to the old server if something catastrophic happens, and see how the new server performs.

Improve this answer
2
  • i'll do that tomorrow, but i thought asking for an opinion from more experienced people would help me by doing at least the basic setup right from start and not having to re-do things that would cause downtime if something is misconfigured badly. wouldn't raid5/6 be an option too? is the difference that big? thanks
    – user27056
    Nov 24, 2009 at 6:18
  • This might be worth reading: blogs.zdnet.com/Ou/?p=484 And I assume you've perused this: en.wikipedia.org/wiki/RAID
    – molecularbear
    Nov 24, 2009 at 13:14
1

You're telling us that you think you should be capable of supporting 300-400Mbps, but you can't afford to use CDN? Something is fishy... First of all, you have some uplink connection... what is that connection? what speed does it support? Is the bandwidth dedicated or shared? I would guess that this is the first bottleneck in your setup far far before I ever would think it is the server.

FYI, carrier grade datacenter badwidth is around $250 /Mbps /mo continuous, so even at sustained 100Mbps levels you are talking about $25k/mo in expenses. If you are doing that you can afford to look at a CDN.

Improve this answer
1
  • No, connection is not the bottleneck and we pay waaay less then that, it's called a MAN (metropolitan area network) which most of our ISP consider it country wide and 90% of our users come from here. i pushed 2gbps live streaming with that server by paying ~1.2k euros/month so again, i don't think CDN is an option in this case because the price difference is pretty high. So, i have a server issue as i asked in the first place, not a connection one as i can theoretically do ~4gbps now across all our servers and it's upgradeable, try giving people a bit of benefit of the doubt, thanks :)
    – user27056
    Nov 24, 2009 at 6:06

You must log in to answer this question.