Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have this script from Colin Johnson on Github - https://github.com/colinbjohnson/aws-missing-tools/tree/master/ec2-automate-backup

It seems great. I have modified it to send email to myself every time an EBS snapshot is created or deleted. The following works like a charm

ec2-automate-backup.sh -v "vol-myvolumeid" -k 3

However, it does not execute at all as part of my crontab (I didn't receive any emails)

#some command that got commented out

*/5 * * * * ec2-automate-backup.sh -v "vol-fb2fbcdf" -k 3;

* * * * * date >> /root/logs/crontab.log;

*/5 * * * * date >> /root/logs/crontab2.log

Please note that the 2nd and 3rd execute just fines as I can see the date and time in log files.

What could I have missed here?

The full ec2-automate-backup.sh is as follows:

#!/bin/bash -
# Author: Colin Johnson / colin@cloudavail.com
# Date: 2012-09-24
# Version 0.1
# License Type: GNU GENERAL PUBLIC LICENSE, Version 3
#
#confirms that executables required for succesful script execution are available

prerequisite_check()
{
    for prerequisite in basename ec2-create-snapshot ec2-create-tags ec2-describe-snapshots ec2-delete-snapshot date
    do
        #use of "hash" chosen as it is a shell builtin and will add programs to hash table, possibly speeding execution. Use of type also considered - open to suggestions.
        hash $prerequisite &> /dev/null
        if [[ $? == 1 ]] #has exits with exit status of 70, executable was not found
            then echo "In order to use `basename $0`, the executable \"$prerequisite\" must be installed." 1>&2 | mailx -s "Error happened 0" eric@mydomain.com ; exit 70
        fi
    done
}

#get_EBS_List gets a list of available EBS instances depending upon the selection_method of EBS selection that is provided by user input
get_EBS_List()
{
    case $selection_method in
        volumeid)
            if [[ -z $volumeid ]]
                then echo "The selection method \"volumeid\" (which is $app_name's default selection_method of operation or requested by using the -s volumeid parameter) requires a volumeid (-v volumeid) for operation. Correct usage is as follows: \"-v vol-6d6a0527\",\"-s volumeid -v vol-6d6a0527\" or \"-v \"vol-6d6a0527 vol-636a0112\"\" if multiple volumes are to be selected." 1>&2 | mailx -s "Error happened 1" eric@mydomain.com ; exit 64
            fi
            ebs_selection_string="$volumeid"
            ;;
        tag) 
            if [[ -z $tag ]]
                then echo "The selected selection_method \"tag\" (-s tag) requires a valid tag (-t key=value) for operation. Correct usage is as follows: \"-s tag -t backup=true\" or \"-s tag -t Name=my_tag.\"" 1>&2 | mailx -s "Error happened 2" eric@mydomain.com ; exit 64
            fi
            ebs_selection_string="--filter tag:$tag"
            ;;
        *) echo "If you specify a selection_method (-s selection_method) for selecting EBS volumes you must select either \"volumeid\" (-s volumeid) or \"tag\" (-s tag)." 1>&2 | mailx -s "Error happened 3" eric@mydomain.com ; exit 64 ;;
    esac
    #creates a list of all ebs volumes that match the selection string from above
    ebs_backup_list_complete=`ec2-describe-volumes --show-empty-fields --region $region $ebs_selection_string 2>&1`
    #takes the output of the previous command 
    ebs_backup_list_result=`echo $?`
    if [[ $ebs_backup_list_result -gt 0 ]]
        then echo -e "An error occured when running ec2-describe-volumes. The error returned is below:\n$ebs_backup_list_complete" 1>&2 | mailx -s "Error happened 4" eric@mydomain.com ; exit 70
    fi
    ebs_backup_list=`echo "$ebs_backup_list_complete" | grep ^VOLUME | cut -f 2`
    #code to right will output list of EBS volumes to be backed up: echo -e "Now outputting ebs_backup_list:\n$ebs_backup_list"
}

create_EBS_Snapshot_Tags()
{
    #snapshot tags holds all tags that need to be applied to a given snapshot - by aggregating tags we ensure that ec2-create-tags is called only onece
    snapshot_tags=""
    #if $name_tag_create is true then append ec2ab_${ebs_selected}_$date_current to the variable $snapshot_tags
    if $name_tag_create
        then
        ec2_snapshot_resource_id=`echo "$ec2_create_snapshot_result" | cut -f 2`
        snapshot_tags="$snapshot_tags --tag Name=ec2ab_${ebs_selected}_$date_current"
    fi
    #if $purge_after_days is true, then append $purge_after_date to the variable $snapshot_tags
    if [[ -n $purge_after_days ]]
        then
        snapshot_tags="$snapshot_tags --tag PurgeAfter=$purge_after_date --tag PurgeAllow=true"
    fi
    #if $snapshot_tags is not zero length then set the tag on the snapshot using ec2-create-tags
    if [[ -n $snapshot_tags ]]
        then echo "Tagging Snapshot $ec2_snapshot_resource_id with the following Tags:"
        ec2-create-tags $ec2_snapshot_resource_id --region $region $snapshot_tags
        #echo "Snapshot tags successfully created" | mailx -s "Snapshot tags successfully created" eric@mydomain.com
    fi
}

date_command_get()
{
    #finds full path to date binary
    date_binary_full_path=`which date`
    #command below is used to determine if date binary is gnu, macosx or other
    date_binary_file_result=`file -b $date_binary_full_path`
    case $date_binary_file_result in
        "Mach-O 64-bit executable x86_64") date_binary="macosx" ;;
        "ELF 64-bit LSB executable, x86-64, version 1 (SYSV)"*) date_binary="gnu" ;;
        *) date_binary="unknown" ;;
    esac
    #based on the installed date binary the case statement below will determine the method to use to determine "purge_after_days" in the future
    case $date_binary in
        gnu) date_command="date -d +${purge_after_days}days -u +%Y-%m-%d" ;;
        macosx) date_command="date -v+${purge_after_days}d -u +%Y-%m-%d" ;;
        unknown) date_command="date -d +${purge_after_days}days -u +%Y-%m-%d" ;;
        *) date_command="date -d +${purge_after_days}days -u +%Y-%m-%d" ;;
    esac
}

purge_EBS_Snapshots()
{
    #snapshot_tag_list is a string that contains all snapshots with either the key PurgeAllow or PurgeAfter set
    snapshot_tag_list=`ec2-describe-tags --show-empty-fields --region $region --filter resource-type=snapshot --filter key=PurgeAllow,PurgeAfter`
    #snapshot_purge_allowed is a list of all snapshot_ids with PurgeAllow=true
    snapshot_purge_allowed=`echo "$snapshot_tag_list" | grep .*PurgeAllow'\t'true | cut -f 3`

    for snapshot_id_evaluated in $snapshot_purge_allowed
    do
        #gets the "PurgeAfter" date which is in UTC with YYYY-MM-DD format (or %Y-%m-%d)
        purge_after_date=`echo "$snapshot_tag_list" | grep .*$snapshot_id_evaluated'\t'PurgeAfter.* | cut -f 5`
        #if purge_after_date is not set then we have a problem. Need to alter user.
        if [[ -z $purge_after_date ]]
            #Alerts user to the fact that a Snapshot was found with PurgeAllow=true but with no PurgeAfter date.
            then echo "A Snapshot with the Snapshot ID $snapshot_id_evaluated has the tag \"PurgeAllow=true\" but does not have a \"PurgeAfter=YYYY-MM-DD\" date. $app_name is unable to determine if $snapshot_id_evaluated should be purged." 1>&2 | mailx -s "Error happened 5" eric@mydomain.com
        else
            #convert both the date_current and purge_after_date into epoch time to allow for comparison
            date_current_epoch=`date -j -f "%Y-%m-%d" "$date_current" "+%s"`
            purge_after_date_epoch=`date -j -f "%Y-%m-%d" "$purge_after_date" "+%s"`
            #perform compparison - if $purge_after_date_epoch is a lower number than $date_current_epoch than the PurgeAfter date is earlier than the current date - and the snapshot can be safely removed
            if [[ $purge_after_date_epoch < $date_current_epoch ]]
                then
                echo "The snapshot \"$snapshot_id_evaluated\" with the Purge After date of $purge_after_date will be deleted."
                ec2-delete-snapshot --region $region $snapshot_id_evaluated
                echo "Old snapshots successfully deleted for $volumeid" | mailx -s "Old snapshots successfully deleted for $volumeid" eric@mydomain.com
            fi
        fi
    done
}

#calls prerequisitecheck function to ensure that all executables required for script execution are available
prerequisite_check

app_name=`basename $0`

#sets defaults
selection_method="volumeid"
region="ap-southeast-1"
#date_binary allows a user to set the "date" binary that is installed on their system and, therefore, the options that will be given to the date binary to perform date calculations
date_binary=""

#sets the "Name" tag set for a snapshot to false - using "Name" requires that ec2-create-tags be called in addition to ec2-create-snapshot
name_tag_create=false
#sets the Purge Snapshot feature to false - this feature will eventually allow the removal of snapshots that have a "PurgeAfter" tag that is earlier than current date
purge_snapshots=false
#handles options processing
while getopts :s:r:v:t:k:pn opt
    do
        case $opt in
            s) selection_method="$OPTARG";;
            r) region="$OPTARG";;
            v) volumeid="$OPTARG";;
            t) tag="$OPTARG";;
            k) purge_after_days="$OPTARG";;
            n) name_tag_create=true;;
            p) purge_snapshots=true;;
            *) echo "Error with Options Input. Cause of failure is most likely that an unsupported parameter was passed or a parameter was passed without a corresponding option." 1>&2 ; exit 64;;
        esac
    done

#sets date variable
date_current=`date -u +%Y-%m-%d`
#sets the PurgeAfter tag to the number of days that a snapshot should be retained
if [[ -n $purge_after_days ]]
    then
    #if the date_binary is not set, call the date_command_get function
    if [[ -z $date_binary ]]
        then date_command_get
    fi
    purge_after_date=`$date_command`
    echo "Snapshots taken by $app_name will be eligible for purging after the following date: $purge_after_date."
fi

#get_EBS_List gets a list of EBS instances for which a snapshot is desired. The list of EBS instances depends upon the selection_method that is provided by user input
get_EBS_List

#the loop below is called once for each volume in $ebs_backup_list - the currently selected EBS volume is passed in as "ebs_selected"
for ebs_selected in $ebs_backup_list
do
    ec2_snapshot_description="ec2ab_${ebs_selected}_$date_current"
    ec2_create_snapshot_result=`ec2-create-snapshot --region $region -d $ec2_snapshot_description $ebs_selected 2>&1`
    if [[ $? != 0 ]]
        then echo -e "An error occured when running ec2-create-snapshot. The error returned is below:\n$ec2_create_snapshot_result" 1>&2 ; exit 70
    else
        ec2_snapshot_resource_id=`echo "$ec2_create_snapshot_result" | cut -f 2`
        echo "Snapshots successfully created for volume $volumeid" | mailx -s "Snapshots successfully created for $volumeid" eric@mydomain.com
    fi  
    create_EBS_Snapshot_Tags
done

#if purge_snapshots is true, then run purge_EBS_Snapshots function
if $purge_snapshots
    then echo "Snapshot Purging is Starting Now."
    purge_EBS_Snapshots
fi

cron log

Oct 23 10:24:01 ip-10-130-153-227 CROND[28214]: (root) CMD (root (ec2-automate-backup.sh -v "vol-myvolumeid" -k 3;))
Oct 23 10:24:01 ip-10-130-153-227 CROND[28215]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:25:01 ip-10-130-153-227 CROND[28228]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:25:01 ip-10-130-153-227 CROND[28229]: (root) CMD (date >> /root/logs/crontab2.log)
Oct 23 10:26:01 ip-10-130-153-227 CROND[28239]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:27:01 ip-10-130-153-227 CROND[28247]: (root) CMD (root (ec2-automate-backup.sh -v "vol-myvolumeid" -k 3;))
Oct 23 10:27:01 ip-10-130-153-227 CROND[28248]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:28:01 ip-10-130-153-227 CROND[28263]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:29:01 ip-10-130-153-227 CROND[28275]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:30:01 ip-10-130-153-227 CROND[28292]: (root) CMD (root (ec2-automate-backup.sh -v "vol-myvolumeid" -k 3;))
Oct 23 10:30:01 ip-10-130-153-227 CROND[28293]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:30:01 ip-10-130-153-227 CROND[28294]: (root) CMD (date >> /root/logs/crontab2.log)
Oct 23 10:31:01 ip-10-130-153-227 CROND[28312]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:32:01 ip-10-130-153-227 CROND[28319]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:33:01 ip-10-130-153-227 CROND[28325]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:33:01 ip-10-130-153-227 CROND[28324]: (root) CMD (root (ec2-automate-backup.sh -v "vol-myvolumeid" -k 3;))
Oct 23 10:34:01 ip-10-130-153-227 CROND[28345]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:35:01 ip-10-130-153-227 CROND[28362]: (root) CMD (date  >> /root/logs/crontab.log;)
Oct 23 10:35:01 ip-10-130-153-227 CROND[28363]: (root) CMD (date >> /root/logs/crontab2.log)

Mails to root

From root@ip-10-130-153-227.ap-southeast-1.compute.internal  Tue Oct 23 06:00:01 2012
Return-Path: <root@ip-10-130-153-227.ap-southeast-1.compute.internal>
Date: Tue, 23 Oct 2012 06:00:01 GMT
From: root@ip-10-130-153-227.ap-southeast-1.compute.internal (Cron Daemon)
To: root@ip-10-130-153-227.ap-southeast-1.compute.internal
Subject: Cron <root@ip-10-130-153-227> root ec2-automate-backup.sh -v "vol-myvolumeid" -k 3
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Status: R

/bin/sh: root: command not found

Update Apparently, undx is right about cron having limited environment (x-cron-env) The backup script is located in /usr/local/bin/ which is not included in PATH of x-cron-env. This opens a new horizon to me as I assume cron simply execute the shell script the same way the script executes itself. Off to making the AWS CLI Tools work with cron now

As Colin Johnson has updated his script and as some of you pointed out, there was actually no need for me to edit such script. I simply needed to use it properly (with proper understanding of CRON and a bit of AWS CLI Tools). Doesn't make sense for amateur like myself to attempt to change the awesome script by guru like Colin either.

The backup script has worked beautifully me ever soon. Strong recommend AWS Missing Tools to everyone.

share|improve this question
    
Does the job show up in the cron log? Does anything show up in the default mail directory (/var/spool/mail/<USERNAME>)? –  Ladadadada Oct 23 '12 at 10:28
    
Yes, sir :) let me update my question –  fuzzybee Oct 23 '12 at 10:33
1  
I've since updated ec2-automate-backup to have a "cron-primer" option to make configuration much easier - simply run ec2-automate-backup with "-c path_to_cron_primer" file to provide only the environment configuration necessary for running under cron. –  Colin Johnson Nov 12 '12 at 3:33
add comment

3 Answers

up vote 10 down vote accepted

On a default UNIX-like system, cron has a minimal environment defined. Usually HOME, SHELL, LOGNAME are defined and PATH is set to /bin.

You have two solutions:

  • enter the full path of your script. ie: /home/me/bin/ec2-automate-backup.sh
  • alter the PATH environment variable.

If you want to receive email from the cron daemon define the MAILTO variable.

MAILTO=me@example.org
PATH=/bin:/home/me/bin
*/5 * * * * ec2-automate-backup.sh -v "vol-fb2fbcdf" -k 3
# OR
*/5 * * * * /path/to/script/ec2-automate-backup.sh -v "vol-fb2fbcdf" -k 3
share|improve this answer
    
As mentionned in your update in Mails to root Your PATH is X-Cron-Env: <PATH=/usr/bin:/bin>. Where is located your script ? –  undx Oct 23 '12 at 10:47
    
My script is at /usr/local/bin/ec2-automate-backup.sh. You are absolutely right about cron having limited environment defined. Having full path to the script does get my script executed. Thank you very much! –  fuzzybee Oct 23 '12 at 12:53
add comment

executable ? permissions ? Don't forget that since they are executed inside cron, they won't be executed as your normal user (so you may have some permission problems)

what if you change the line to

*/5 * * * * root (ec2-automate-backup.sh -v "vol-fb2fbcdf" -k 3;)
share|improve this answer
    
Hi @Nikolaidis, the file has permission 755. I've tried having the username root there in crontab but no success –  fuzzybee Oct 23 '12 at 10:32
    
That runs it in a subshell; how is that an improvement ? –  adaptr Oct 23 '12 at 12:32
add comment

Your cron script starts with #!/bin/bash -; however, this is ignored by cron.

Cron executes all commands with /bin/sh.

If you wish to execute a bash script with cron, you need to invoke it directly with bash:

*/5 * * *  * /bin/bash /full/path/to/your/script

Your script uses bash-specific features which will fail when run with the standard Bourne sh(1).

There are numerous issues with the script you present; I would strongly advise you to study the BashGuide for expert advice on how to craft efficient and safe bash scripts.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.