About using cron in Linux to schedule my python program

Question

I want to use cron to hourly run a python program in Linux(ubuntu). I wrote a script called script.sh

cd Dropbox/NetworkProject/AMT_Crawler/
scrapy crawl AmtCrawler --set FEED_URI=data.json --set FEED_FORMAT=json

Then I used

crontab -e

and add a line like this

*/30 * * * * sh Dropbox/NetworkProject/AMT_Crawler/script.sh 2>&1 >> /Dropbox/NetworkProject/AMT_Crawler/output.log

After this, I run

sudo /etc/init.d/cron start

in terminal. It said

Rather than invoking init scripts through /etc/init.d, use the service(8) utility, e.g. service cron start Since the script you are attempting to invoke has been converted to an Upstart job, you may also use the start(8) utility, e.g. start cron

So I run this

service cron start

Then there was an error:

start: Rejected send message, 1 matched rules; type="method_call", sender=":1.196" (uid=1000 pid=12574 comm="start cron ") interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)" requested_reply="0" destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init")

Now I'm stuck with this problem asking for some help.

Answer 1

cron(8) is always running. You do not need to start or restart it when you make changes to your crontab(5) file using crontab -e:

   Additionally, cron checks each minute to see if its spool
   directory's modtime (or the modtime on /etc/crontab) has
   changed, and if it has, cron will then examine the modtime on
   all crontabs and reload those which have changed.  Thus cron
   need not be restarted whenever a crontab file is modified.
   Note that the crontab(1) command updates the modtime of the
   spool directory whenever it changes a crontab.

(The last sentence is the reason why it is always recommended to use the crontab(1) program when modifying your own crontab(5) file.)

Answer 2

I don't know whether this is the whole problem, but your output redirections are reversed. Rather than:

command 2>&1 >> /path/to/output.log

(which redirects stderr to stdout and then redirects stdout, but not stderr to output.log), you need:

command >> /path/to/output.log 2>&1

As others have said, the service cron start command shouldn't be necessary; crond should already be running (as you've already confirmed by getting an empty log file).

Answer 3

Try sudo service cron start.

Answer 4

*/30 * * * * sh Dropbox/NetworkProject/AMT_Crawler/script.sh 2>&1 >> /Dropbox/NetworkProject/AMT_Crawler/output.log

You need the full path for your dropbox directory. It is not under /.

Try this (assuming the Dropbox directory is $HOME/Dropbox):

*/30 * * * * sh $HOME/Dropbox/NetworkProject/AMT_Crawler/script.sh >> $HOME/Dropbox/NetworkProject/AMT_Crawler/output.log 2>&1

Note that this changes both references to the Dropbox directory. It also fixes the ordering of the redirection operators; see my (Keith Thompson's) answer.