Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We use WhatsUp Gold to monitor all of our web servers. On our Linux servers (and much to the same degree, our FreeBSD servers) I'm having a little bit of an issue with the memory monitors. We're using SNMP with WUG to grab the data from the servers. The memory counter that the SNMP daemon returns on the servers is the combined value (used, cached, buffers). Right now one of my servers looks like this:

[admin@stgwww snmp]$ free -m
             total       used       free     shared    buffers     cached 
Mem:          7872       1656       6216          0        143      1107
-/+ buffers/cache:        404       7467 
Swap:         4867          0       4867

The Value being returned via SNMP to WUG is 1656. From what I understand, the cached RAM is essentially FREE RAM with the added benefit of hanging on to data that previously occupied it in case it's needed again. So for our purposes of wanting to know how much RAM is actually being actively used, the value we're getting back is misleading. If we go off of what's being graphed by WUG, we're being led to believe that more RAM is being used and less is available than there really is.

So whats that best way to go about monitoring this? WUG allows me to write SSH scripts, which can SSH into the server every 5 minutes or so, execute a script and return the value (as long as it's a single numeric value). With this I've written a script that pulls the "404" number from the example above and divides it by the total amount giving me a percent used value which I return to WUG and graph on a chart that scales from 0 to 100. But this seems like way to much of a hack.

Am I better off monitoring the free+buffers+cached value? Is there a better way to do this in WUG? Thoughts?

share|improve this question
    
Yes, 404/7872*100 is what I'd call the correct number for memory usage. Terminology will really get you with memory, so make sure everyone knows what your number actually means. –  pboin Dec 27 '11 at 18:01
    
Safado, I'm facing the same problem here. Would you mind to give me some piece of advice on this matter? Can I have a copy of your SSH script so I can use it with WUG? Regards, Renan PS! - I would like to post this as a comment, but I don't have enough reputation. Moreover, I couldn't find a way to contact Safado directly. –  Renan Mar 20 at 19:54
    
@Renan - I posted an answer with our solution we've been using. Hope it gives you what you need. –  Safado Apr 1 at 20:44

4 Answers 4

up vote 5 down vote accepted

Go and take a look at linuxatemyram.com. WUG is telling you what Linux thinks is used (used+buffers+cache). What you have decided to monitor (used/total) seems reasonable to me especially for a graph as it requires no knowledge of the system specifics.

share|improve this answer
    
Looks like I'll stick to the custom script idea. I looked through the MIB to see all the values being passed over and it only has Used+buffers+cache, Virtual, Buffers, Cached, and Swap. I was hoping that WUG would allow me to do some math, with the data, but it won't. It just seems that creating an SSH connection and destroying it every minute to run a script is so inefficient. Plus I'm only polling every 60 seconds. –  Safado Dec 27 '11 at 20:20

Free ram is free ram and buffers are cached ram which can be reclaimed. Most of the monitoring tools I've used present this difference in an accumulative area graph which presents at least used, cached and inactive memory stacked under the 100% level and swap over these. The only way to have a correct knowledge of how server is performing is to view all of them.

If you only can graph a value I'll recomend to you graph the used memory, and consider 'free' the rest. Oh and i will recomend also switch monitoring tools. Even munin with the default config has a decent memory graph.

share|improve this answer

I recommend ganglia: http://ganglia.sourceforge.net/

It does memory monitoring and divides it into constituent parts. There is almost zero configuration. You install a daemon on each linux box and then designate one central box to record the RRDs.

Here's an example memory graph:

Graph 1

share|improve this answer
    
We also monitor the server through Munin, which appears to be the exact same as ganglia, but it doesn't give us quite the capability we're after. It does, however, provide us Apache and MySQL monitors that WUG doesn't. –  Safado Dec 27 '11 at 20:00

For those (Renan) interested to know what solution I came up with.

I've been using a custom bash script to retrieve the memory (Used / Total) and then converting it to a percentage.

#!/bin/bash                                                                                                                                                                                                                              

USED=`free -m | grep "buffers/cache" | awk '{print $3}'`
TOTAL=`free -m | grep "Mem:" | awk '{print $2}'`

VALUE=`bc -l <<< "scale=2; (${USED}/${TOTAL})*100" | sed 's/\.[0-9][0-9]//'`

exit $VALUE

I then use a custom SNMP counter to execute that script and return the value. In the snmpd.conf file, it looks like this:

exec   check_mem /usr/share/snmp/check_mem.sh

Each exec script returns a few OIDs with things like the name of the script, the exit status, the return value, and so on. The unfortunate part is that the return value is a string and not an integer, so WUG has some problems graphing it (it still graphs it, but the real time graphs won't work). So in this case where we know the value will always be under 100, I set it to the exit status and then I poll that OID.

To monitor it in WUG create a custom SNMP performance monitor and monitor the OID of the exit status of that exec script. You can then create custom alerts and what not.

We've been using it for a while now and it works great. Hope that helps!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.