1

I have been trying to deal with this strage problem all morning.

I have a simple crawler.py file, which contans a code that can only run under python3 to be exact under python3.3 evenrionment

Now, my default CentOs7 machine comes with python2.7 so, I am forced to switch using python3 to run my script, example: 

scl enable python33 bash

example. 

[root@bt py]# python -V
Python 2.7.5
[root@bt py]# scl enable python33 bash
[root@bt py]# python -V
Python 3.3.2

as you can see, it changed the version after which, I can run simple run crawler.py without a problem as easy as python crawler.py.

But, I need this python file to run via cron, at specific time of the week.

The problem is that once I log out of my VPS, python2.7 becomes the default and if cron triggers it, it won't run. For this I thought I should use the python3 binary instead of just python .. so, I tried with the following 

/opt/rh/python33/root/usr/bin/python3.3 crawler.py

But, it seems the binary does not appear unles you run scl enable python33 bash

you can see this for yourself, 

[root@bt ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /usr/share/man/man1/python.1.gz
[root@bt ~]# scl enable python33 bash
[root@bt ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /opt/rh/python33/root/usr/bin/python3.3m /opt/rh/python33/root/usr/bin/python3.3-config /opt/rh/python33/root/usr/bin/python3.3m-config /opt/rh/python33/root/usr/bin/python /opt/rh/python33/root/usr/bin/python3.3 /usr/share/man/man1/python.1.gz

The binary, only appears after python33 is called via scl. So, all the time I have to tell cron to somehow execute scl enable python bash command so my python3 scripts runs without issues.

So, I created a bash file as 

#!/bin/bash 
$(scl enable python33 bash)
python crawler.py 
# alternatively I tried the below solution also. 
# /opt/rh/python33/root/usr/bin/python3.3 crawler.py

But, once bash executes the first command it exits the process or spawns other processes. even tried putting this

scl enable python33 bash && python crawl.py

also 

scl enable python33 bash && /opt/rh/python33/root/usr/bin/python3.3 crawl.py

So, in a nutshell, it seem there is no way to run a python3 script via cronjob, if you have anything but python3 as default in your machine.

Improve this question
  • Why do you keep calling bash instead of the script you really want to run? – Michael Hampton Mar 27 '16 at 18:11
  • @MichaelHampton Ah, if you mean the #!/bin/bash because, the there is an if/else statement, as the actual python files are more than one, so I need something like bash crawl.py abc and bash crawl.py xyz those commands are sent from cron, but in the bash I need to use if/else to indentify the argument .. so, to simply put it, the bash has conditionals. – samayo Mar 27 '16 at 18:15
  • @ANW: See answer to your question: stackoverflow.com/q/36245769/3776858 – Cyrus Mar 27 '16 at 19:31
  • @Cyrus This is completely different, please read this question if you want. – samayo Mar 27 '16 at 19:58
  • To clarify, I accepted that answer too soon, without realizing that /opt/rh/python33/root/usr/bin/python3.3 script.py does not work once you log out – samayo Mar 27 '16 at 20:05
4

If I understand it correctly, then you want to use cron to run a bash script, in which a call to python results in the scl'ed python33 version.

I just implemented and tested that here, on a Scientific Linux box with python33 from scl, just as in your case.

Create a bash script:

$ cat /home/username/mybashfile.sh 
#!/bin/bash
echo `date` >> /home/username/pythonversionfile
python --version >> /home/username/pythonversionfile 2>&1

and create a cron entry like this:

* * * * * /usr/bin/scl enable python33 'bash -c /home/username/mybashfile.sh'

Which means you call scl to use the python33 environment, which calls bash, directly with the call to the script you created before.

This works nicely and gives me the following output:

$ tail pythonversionfile 
Wed Mar 30 17:36:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:37:02 CEST 2016
Python 3.3.2
Wed Mar 30 17:38:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:39:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:40:01 CEST 2016
Python 3.3.2

So you can see that the bash script is called by cron every minute, and inside the bash script the environment is correctly set such that python results in python 3.3.2 being called.

Improve this answer
1
+150

Cronjob (example):

0 1 * * * scl enable python33 'python /path/to/crawl.py'

Syntax of scl:

scl enable software_collection... 'command...'

Source

Improve this answer
  • Yup - just worked this out myself, mildly interesting too. – user9517 Mar 29 '16 at 20:05
  • Well, unfortunately it does not work. I have tried to this before, and tried it now just to be sure, I see nothing. For example, the crawl.py does not seem to get executed. To be sure I did manually scl enable python33 the python crawl.py and I can see this works, and if I paste scl enable python33 'python p.py' and execute it it works, but not from cron. Checked /var/log/cron there is no errors, or permission issues. – samayo Mar 30 '16 at 8:14
  • Try this step: 0 1 * * * scl enable python33 'python --version' >/tmp/version.txt – Cyrus Mar 30 '16 at 8:18
  • You can check this pastebin as you can see, the version.txt gets created but nothing is written on it, simultaneously I get notice from cronjob (guessing!) that says "You have new mail in /var/spool/mail/root" and when I check the mail, I can see the python version. – samayo Mar 30 '16 at 8:28
  • 1
    Okay, python wrote its output to stderr. – Cyrus Mar 30 '16 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.