Memcached Debuging/Server Logs Monitor the Memcached Servers?

Question

I have chat engine which is based on the Memcached variables, putting them into arrays and reading them in other end via jquery,

which works fine 95% of the times, however when the server load is high memcached (presume its the memcached) the crash and browser gets stucks up.

I dont think its jquery issue since this only happens when the server load is very high.

I need a way to monitor the memcached servers or somehow write a log file into where the fails/errors comes in...

Any idea on how i can do this ? or any idea why memcached servers fails ? I run the memcached as follows

$GLOBALS['MemCached'] = FALSE;
$GLOBALS['MemCached'] = new Memcache;
$GLOBALS['MemCached']->pconnect('localhost', 11211);

My memcached config is as follows

#! /bin/sh
#
# chkconfig: - 55 45
# description:  The memcached daemon is a network memory cache service.
# processname: memcached
# config: /etc/sysconfig/memcached
# pidfile: /var/run/memcached/memcached.pid

# Standard LSB functions
#. /lib/lsb/init-functions

# Source function library.
. /etc/init.d/functions

PORT=11211
USER=memcached
MAXCONN=1024
CACHESIZE=128
OPTIONS=""

if [ -f /etc/sysconfig/memcached ];then 
    . /etc/sysconfig/memcached
fi

# Check that networking is up.
. /etc/sysconfig/network

if [ "$NETWORKING" = "no" ]
then
    exit 0
fi

RETVAL=0
prog="memcached"
pidfile=${PIDFILE-/var/run/memcached/memcached.pid}
lockfile=${LOCKFILE-/var/lock/subsys/memcached}

start () {
    echo -n $"Starting $prog: "
    # Ensure that /var/run/memcached has proper permissions
    if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
        chown $USER /var/run/memcached
    fi

    daemon --pidfile ${pidfile} memcached -d -p $PORT -u $USER  -m $CACHESIZE -c $MAXCONN -P ${pidfile} $OPTIONS
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch ${lockfile}
}
stop () {
    echo -n $"Stopping $prog: "
    killproc -p ${pidfile} /usr/bin/memcached
    RETVAL=$?
    echo
    if [ $RETVAL -eq 0 ] ; then
        rm -f ${lockfile} ${pidfile}
    fi
}

restart () {
        stop
        start
}


# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  status)
    status -p ${pidfile} memcached
    RETVAL=$?
    ;;
  restart|reload|force-reload)
    restart
    ;;
  condrestart|try-restart)
    [ -f ${lockfile} ] && restart || :
    ;;
  *)
    echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart|try-restart}"
    RETVAL=2
        ;;
esac

exit $RETVAL

Answer 1

Define an additional variable in /etc/sysconfig/memcached:

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
LOGFILE="/var/log/memcached.log"
OPTIONS=""

Edit the start() function in the init script as belows:

start () {
    #echo -n $"Starting $prog: "
    # insure that /var/run/memcached has proper permissions
    if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
        chown $USER /var/run/memcached
    fi

    #daemon --pidfile /var/run/memcached/memcached.pid memcached -d -p $PORT -u $USER  -m $CACHESIZE -c $MAXCONN -P /var/run/memcached/memcached.pid $OPTIONS -vv > $LOGFILE 2>&1
    $prog -d -p $PORT -u $USER  -m $CACHESIZE -c $MAXCONN -P /var/run/memcached/memcached.pid $OPTIONS -vv > $LOGFILE 2>&1
    RETVAL=$?
    #echo
    [ $RETVAL -eq 0 ] && action $"Starting $prog: " /bin/true && touch /var/lock/subsys/memcached
}

then restart the memcached, you'll see something like this in the /var/log/memcached.log:

slab class  40: chunk size    616944 perslab       1
slab class  41: chunk size    771184 perslab       1
slab class  42: chunk size   1048576 perslab       1
<26 server listening (auto-negotiate)
<27 server listening (auto-negotiate)
<28 send buffer was 129024, now 268435456
<29 send buffer was 129024, now 268435456
<28 server listening (udp)
<29 server listening (udp)

Don't forget to configure logrotate.

Answer 2

First, you are not showing memcached configuration, you are showing the startup script. You need to look for a file named like /etc/memcached.conf. To enable logging, you need to uncomment the line

logfile /var/log/memcached.log

Also, you can increase the verbosity by enabling -v or -vv option.

If memcached is crashed, you can look into system logs /var/log/messages and /var/log/syslog to check for possible error conditions.

Answer 3

Considering your memcached config is at [/etc/sysconfig/memcached] (Redhat RHEL & Centos 8 etc) you can just add the following switch to your options and restart memcached. Then it will log stuff into /var/log/messages(add rotation if you like later)

OPTIONS="-vv"

If you don't want too much logging remove one of the v

The following configuration will make memcache listen on localhost and disable udp in addition to logging to /var/log/messages

OPTIONS="-l 127.0.0.1, -U 0, -vv"

After you're satisfied with logging and you don't need logging anymore, just remove everything from options [OPTIONS=""] then use the following commands to check when you need to:

memcached-tool 127.0.0.1:11211 stats

OR

echo stats | nc 127.0.0.1 11211