4

I need some persistent storage across spot terminations.

My approach to solve the problem is

  1. Write a startup script that attaches and mounts the persistent volume

  2. Snapshot this state and register a private AMI

  3. Launch Spot instances based of the private AMI

searches in this forum [1] suggests this is the correct procedure

My startup script:

#!/bin/sh

echo "executing startup script"

# attach the EBS volume to this machine
aws ec2 attach-volume --volume-id vol-7bef1d96  --instance-id  $(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id ) --device /dev/xvdg

sleep  10

# mount the attached EBS volume
echo "mounting the  attached volume"
sudo mount /dev/xvdg1 /home/ubuntu/persistent/

#run script
echo "starting the  dummy program in background"
python2 persistent/dummyProgram.py &

The script works perfectly if executed manually. However when it is executed by cron the only the echo statement are executed

the crontab task is

@reboot /path/to/startupScript.sh

How do I solve this problem?

The corrected crontab entry reads :

USER=username
PATH=content of $PATH
@reboot /path/to/startupScript.sh
1
  • why don't you put this into userdata instead of cron? that's why they are made for: execute some code at startup. By using cron, you expose yourself to path issues that you would probably not have with userdata
    – Tom
    Aug 24, 2015 at 20:34

2 Answers 2

2

The most likely reason why commands would not be running when invoked from a crontab script is that they are not on the PATH environment variable. Crontab entries run with a minimal set of environment variables by default.

1
  • or give the full path for aws, sudo, and python2.
    – Skaperen
    Aug 27, 2015 at 12:54
0

While cron could definitely get the job done, the canonical way to do this is to specify a user-data script that runs when the spot instance is launched. With the AWS CLI's request-spot-instance this is specified in JSON using the --launch-specification argument.

http://docs.aws.amazon.com/cli/latest/reference/ec2/request-spot-instances.html

This section can be a small shell script and allows you to change the volume-id should that ever be necessary instead of hardcoding that into the AMI.

1
  • I tried using user-data; But for some reason I failed to make it work. so I settled for cron I didn,t notice of the hardcoded volume id but as it's not much of a issue I decided to ignore it for now
    – Soumy
    Aug 25, 2015 at 7:54

Not the answer you're looking for? Browse other questions tagged .