2

I am trying to use logrotate to upload log files from my rails and nginx commands once a day. When I run the logrotate command manually the files successfully upload, but it never does it on its own. I have tried using lastaction and also postrotate but neither actually uploads to S3. Is there something trivial that I'm missing?

/home/deploy/www/mysite.com/log/*log {
su
daily
dateext
dateformat _%Y_%m_%d
rotate 3
missingok
dateext
compress
delaycompress
notifempty
copytruncate
sharedscripts
lastaction
  rvmsudo passenger-config reopen-logs;
  /usr/bin/s3cmd sync /home/deploy/www/mysite.com/log/*.gz s3://mysite_com/logs/rails/
endscript
}

EDIT: Below is the corrected way of doing this. So instead of using the lastaction script, call a shell script instead.

lastaction
  ./rails_reload_and_upload.sh
endscript

which contains

#/bin/bash

./reload_rails_logs.sh
/usr/bin/s3cmd --config /home/username/.s3cfg sync /home/deploy/www/mysite.com/log/*.gz s3://mysite_com/logs/rails/

2 Answers 2

3

I'd say it's most likely that either rvmsudo or s3cmd need extra environment variables to run successfully, which aren't set in the context of logrotate, but which are in your interactive shell session. I wouldn't be surprised if rvmsudo isn't available in the restrictive PATH which cron sets.

1
  • 2
    Yup that was the issue. By emulating the cron environment using env -i $SHELL --norc I was able to figure out what configuration parameters I needed to use.
    – John
    Aug 4, 2015 at 18:06
0

If you are running EC2 servers, other than running a sperate bash script, one can add an Instance Profile to EC2 server so that aws s3 rsync does not need access key which cannot be assigned under cron job.

So the following logrotate config runs pretty well if Instance Profile added on the running EC2 server

/home/ubuntu//logs/*.log {
        copytruncate
        daily
        dateext
        rotate 30
        compress
        missingok
        su root root
        lastaction
                HOSTNAME=$(/bin/hostname -f)
                DD=$(/bin/date +%d)
                MM=$(/bin/date +%m)
                YYYY=$(/bin/date +%Y)
                BUCKET="s3://xxx-logs-archive/$YYYY/$MM/$DD/batch/$HOSTNAME/"
                FILE="/home/ubuntu/logs/batch.log-$YYYY$MM$DD.gz"
                /usr/local/bin/aws s3 cp "$FILE" "$BUCKET"
        endscript
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .