Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

What's the best way to run beanstalkd on an instance of Amazon Linux in EC2? Beanstalkd doesn't exist in the yum repository and the upstart script included with beanstalkd doesn't seem to work out of the box. I'm wondering if someone has already solved this problem, or if I need to go and muck around with a custom EC2 user-data script + beanstalkd upstart script.

share|improve this question
1  
yum search --enablerepo=epel beanstalkd should turn it up on Amazon Linux. –  ceejayoz Sep 8 '13 at 22:08
    
Thanks, this is helpful! Unfortunately, the version is 1.4.6 and there was a critical bug fixed in one of the more recent versions that I ran into. Any other secret repositories that might have a more up to date version? –  Mr. T Sep 8 '13 at 22:23

1 Answer 1

up vote 0 down vote accepted

I ended up writing a user data script that runs when an ec2 instance is created. The script pulls down the latest source for beanstalkd, compiles, and installs the binary. The script uses a modified upstart script to control the lifecycle of the beanstalkd process.

Here's the user data script:

#!/bin/bash -ex
set -e -x

# Log output
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# Update yum first
yum groupinstall "Development Tools" -y 
yum update -y

# Download /etc/default/beanstalkd
curl -J -O https://gist.github.com/phaitour/6482469/download
tar -zxvf *.tar.gz
/bin/cp -f gist*/beanstalkd /etc/default/ 
ln -s /etc/default/beanstalkd /etc/beanstalkd.conf 
rm -rf gist*

# Download /etc/init/beanstalkd.conf
curl -J -O https://gist.github.com/phaitour/6482467/download
tar -zxvf *.tar.gz
/bin/cp -f gist*/beanstalkd.conf /etc/init/
rm -rf gist*

# Create beanstalkd user
/usr/sbin/useradd -c "beanstalk user" -s /bin/false -r -m -d /var/lib/beanstalkd/ beanstalkd

# Compile latest version of beanstalkd
wget https://github.com/kr/beanstalkd/archive/v1.9.tar.gz
tar -zxvf v1.9.tar.gz
cd beanstalkd-1.9
make
mv beanstalkd /usr/bin/beanstalkd-1.9
cd /usr/bin
rm -rf beanstalkd
ln -s beanstalkd-1.9 beanstalkd

# Start beanstalks with upstart
initctl reload-configuration
initctl start beanstalkd

The two scripts it's downloading from gist is the upstart beanstalkd script and a configuration file which sets the options used when running the program.

Here's the upstart beanstalkd.conf script:

description "simple, fast work queue"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 5 2

expect fork

script
    . /etc/default/beanstalkd
    exec su -c "exec /usr/bin/beanstalkd $BEANSTALKD_OPTIONS"
end script

Here's the configuration file copied into /etc/default/beanstalkd:

# Set some constants to be used to construct BEANSTALKD_OPTIONS

BEANSTALKD_ADDR=0.0.0.0
BEANSTALKD_PORT=11300
BEANSTALKD_USER=beanstalkd
BEANSTALKD_BINLOG_DIR=/var/lib/beanstalkd
BEANSTALKD_BINLOG_FSYNC_PERIOD=0



# Create the actual BEANSTALKD_OPTIONS string
# Copied from https://gist.github.com/shiki/515422

BEANSTALKD_OPTIONS="-l ${BEANSTALKD_ADDR} -p ${BEANSTALKD_PORT} -u ${BEANSTALKD_USER}"
if [ "${BEANSTALKD_MAX_JOB_SIZE}" != ""  ]; then
    BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -z ${BEANSTALKD_MAX_JOB_SIZE}"
fi

if [ "${BEANSTALKD_BINLOG_DIR}" != "" ]; then
    if [ ! -d "${BEANSTALKD_BINLOG_DIR}" ]; then
        echo "Creating binlog directory (${BEANSTALKD_BINLOG_DIR})"
        mkdir -p ${BEANSTALKD_BINLOG_DIR} && chown ${BEANSTALKD_USER}:${BEANSTALKD_USER} ${BEANSTALKD_BINLOG_DIR}
    fi
    BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -b ${BEANSTALKD_BINLOG_DIR}"
    if [ "${BEANSTALKD_BINLOG_FSYNC_PERIOD}" != "" ]; then
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -f ${BEANSTALKD_BINLOG_FSYNC_PERIOD}"
    else
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -F"
    fi
    if [ "${BEANSTALKD_BINLOG_SIZE}" != "" ]; then
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -s ${BEANSTALKD_BINLOG_SIZE}"
    fi
fi
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.