Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We've taken over a legacy system that was implemented between 2007-2009 in Ruby and MySQL and since then has had only maintenance work done on it that was necessary to keep it alive.

As a result, the system is now very fragile in terms of its code base. In fact, since half the code base is in one version of Ruby and the other half is in another (this is what our Ruby guy tells us), we have been having problems making any modifications and deploying to production.

Many of the shortcuts taken have made parts of the system unusable - such as some administration queries taking several minutes to complete, or even times out.

We are currently in the process of rewriting the platform in .NET but we need to assess with as much specificity as possible how much the current setup can hold.

One heuristic that raised a red flag for me is that the current production MySQL database is 15GB at the moment and still phases growth because of automatically scheduled sync jobs that write huge audit logs on the database!

Linux reports (through /proc/cpuinfo) 3xXeon E5420 @ 2.5GHz processors at constant 30-50% load when not running nightly jobs. 6GB of RAM + 2GB swap, out of which only 35MB of mem is free and most of swap is free.

The question is: how can we assess in as much detail as possible how much the current setup will hold? This obviously must be done without doing say, a load test, which could bring the server down for good. What heuristics can use to produce an estimate?

Are MySQL databases known to hold databases as large as 15GB+? Is there a known point where reliability is expected to drop significantly?

Any other insights into the investigation of the problem are welcome.

share|improve this question

1 Answer 1

up vote 0 down vote accepted

I think your question about MySQL scalability is a duplicate of this question:

http://stackoverflow.com/questions/1276/how-big-can-a-mysql-database-get-before-performance-starts-to-degrade

15GB is not large at all for a MySQL database. But that doesn't mean that a poorly written application won't run slowly against that database. Terabyte sized MySQL databases are not uncommon - the maximum size of an InnoDB table is 64TB

If it's only grown to 15GB in the past 6 years, I wouldn't expect database growth to be a limiting factor in the time it would take to rewrite the app. The app will probably run about as fast with a 30GB database as it does with a 15GB database

But, to answer your question, if you're running around 30 - 50% load normally, unless you have any severe I/O bottlenecks (like you probably due during the sync jobs that create a lot of audit logs in the database), I'd guess that you can probably support around twice your normal load. It's really a guess, since it depends highly on what your app is actually doing like how much is it reading/writing to the database, how effcient the database queries are (like, are the right indexes in place), and how complicated the application logic is.

The low "free" memory is likely nothing to be concerned with - depending on what utility you used to find "free" memory, "free" doesn't always mean "free". Linux buffers disk data in otherwise unused memory and over time, this buffer cache will grow quite large and consume most of the RAM not used by programs. However, this memory is readily available when needed, the kernel can immediately drop a cache page from RAM anytime it's needed by software. For a better sense of how much free memory you have, take the total memory and subtract "used".

If you really did have only 35MB of free memory left, I think you'd be doing some serious swapping and performance would be quite bad. In that case you'd want to do some MySQL and/or web server tuning to reduce the memory demand. But I don't think that's the situation you're in or you'd be reporting more performance problems.

share|improve this answer
    
Thanks. You didn't however make any comments regarding the high mem usage. Is that safe? If I'm not mistaken Linux will consume all RAM before going to swap for active processes but only 50MB free RAM sounds pretty dangerous to me. Thoughts? –  georgiosd Apr 6 '13 at 15:24
    
I edited my answer to address the memory question –  Johnny Apr 6 '13 at 21:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.