up vote 0 down vote favorite
share [g+] share [fb]

I recently started receiving pubsub data feed from etsy. It averages around 250 notifications per minute. But obviously, when the USA wakes up that spikes quite heavily.

I want to be able to deal with those spikes (about 3 per day) but the rest of day is fine.

What's the best method of getting the right server configuration. My current approach is to keep upgrading until the server stops dying... next leap is:

  • Processor: AMD Phenom II X6-1055T HEXA Core
  • RAM: 4GB DDR2 SDRAM
  • HD1: SATA Drive (7,200 rpm) (+500 GB 7200 RPM SATA hard drive)
  • HD2: SATA Backup Drive (+500 GB SATA (7,200 rpm))
  • OS: Linux OS (+CentOS 5 64-bit)
  • Bandwidth: 6000GB Monthly Transfer (3000 in + 3000 out) (+100M uplink port)

What's the best approach to working out what sort of server setup you need?

Edit: Server load hits 150 during the spikes. It's not a drive space or bandwidth issue.

Edit 2: Thanks for the debug pointers. I have now run SAR and pasted the output here: http://pastebin.com/xNbL91Wp

In summary:

sar 

05:00:01 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
Average:          all     16.74      0.16      2.94      1.51      0.00     78.65


sar -W
12:00:02 AM  pswpin/s pswpout/s
Average:         0.00      0.00


sar -q
12:00:02 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
Average:            7       195      2.20      2.41      2.63

Also, here is a link to a 3 day graph of my server load Server Load over 3 days

The times in the SAR reports and on this graph are both GMT -5

So with a ldavg-1 of 2.20 would you say that a processor upgrade is advisable?

link|improve this question
40% accept rate
1  
I guess that would depend on why the server is dying during those peaks. Are you running at 99% CPU Utilization during that time? Or using up all your bandwith? Or is it just that the HD cannot keep up? Do you have SAR setup on your server, so you can see the bottlenecks? its great when you have it run every 5 min. – Brian Mar 4 '11 at 15:36
1  
Yo uare cheap. RAM is too low today. You can get micro atx boards with 16gb and it costs nothing. I would fill up all the ram and also use a x6-1100. Does your linux support core parking? – TomTom Mar 4 '11 at 15:47
Echoing @Brian and @Sam: High load average doesn't necessarily mean the CPU is the bottleneck. Definitely use SAR (best way to set this up depends on your OS--on Ubuntu, I install the sysstat package) to get some metrics during the spikes. – daveadams Mar 4 '11 at 15:49
1  
@ed209 install sysstat: howtoforge.com/sysstat_monitoring_centos Seems to be about the same on newer versions of CentOS as well. – Brian Mar 4 '11 at 20:03
1  
Hm, more memory could be used to cache pregenerated results, you know. The rule when doing web performance is always to cache as much as you can AS EARLY AS YOU CAN. Check drive io. could be a reason for the spike. – TomTom Mar 6 '11 at 11:30
show 5 more comments
feedback

1 Answer

up vote 3 down vote

Why is you server falling over? This is the first thing you need to establish before you go for an upgrade. Whilst you can keep upgrading your whole server until it works, it'll cost you alot of money and when it stops working again in future your not going to know how to fix it other than throwing more money at it.

Run some performance analysis on the server when these spikes happen to see what is causing you problems. Are you running out of memory, is your CPU at 100% is your disk not able to keep up.

Before you try and fix the problem, find out what your actual problem is, then fix it.

link|improve this answer
Now that I have provided more information, are you able to update your answer with what hardware I should update. – ed209 Mar 6 '11 at 10:23
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.