Am I using too much memory? (Rails on EC2 with Resque)

Question

I am looking at the memory usage of the Rails application (it uses background processes via Resque) and since the common answer to the question, "how many workers is too many" was "test and see", I ran some memory commands and wonder if someone can help figuring if the memory usage is high enough already, or I can still add some extra workers..

so (this is all under the maximum load):

$ free -t -m
                 total       used       free     shared    buffers     cached
    Mem:          1756       1532        223          0         12        229
    -/+ buffers/cache:       1291        464
    Swap:          895         10        885
    Total:        2652       1543       1108

$ vmstat
    procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
     r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
     0  0  10588 156172  13400 326476    1    6     4     0    5    4  1  0 99  0

If there is any extra info I can provide to help answer this, I would be happy to do so. If the question is strange in some way, please let me know I'd be glad to fix etc..

Answer 1

Obviously, if you're actively swapping, you have too many. Swap usage doesn't mean you're actively swapping. Swap I/O means you're actively swapping.

However, actively swapping isn't the only way excessive application memory use can hurt you. Every byte of memory that applications are actively using is one byte of memory that can't be used as a disk cache. If your application doesn't touch the disk very much, this won't matter. But if it does, you can suffer very poor performance due to a low cache hit rate even if there's no swapping.

The first thing to look at is how much memory is either free or being used as a cache. If that's too low, then you risk swapping. In this case, that's about 400MB. So you're not at risk for swapping.

The second thing to look at is whether the disk cache is adequate for your application. One sticky point is whether to count the free memory as part of the disk cache. In your case, the free memory is probably needed for memory churn. That is, it can't be used as disk cache because the cache is constantly getting squeezed as application memory usage increases and decreases. So your disk cache is around 200MB.

Whether that's enough depends on how your application uses the disk. But if it's not, the giveaway will be excessive disk I/O and poor performance.

Answer 2

You should check the file /proc/meminfo. It contains detailed answers of what you are looking for. Keep this in mind that cache and buffer are essentially free memory, barring dirty pages in cache.

When a file is read from disk, it is cached into memory. There exists two copies of the same file in system, one is cached and the other one in disk. Hence, when there is a need to reclaim memory, it is easier to clean cache than to to clean slab memory or swap.

In your scenario, there is a little of swap. Swapping is not bad unless it goes to the situation of thrashing. You don't seem to have a resource crunch on memory and it would be good if you can capture the /proc/meminfo output and architecture of the system when the system usage is at highest level.

Answer 3

From what you indicate I see a little evidence of swapping but not very high at the snapshot. I might stop at the setting you are currently using and back off just a little bit. One thing that you really want to avoid is swapping which is where memory exhaustion is basically indicated.